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1 PROCEE D NGS 


2 THE CLERK: Court calls Criminal Action 21-10104, 


3 United States v. Klyushin. Could counsel and the interpreters 


4 please identify themselves. 


5 MR. FRANK: Stephen Frank and Seth Kosto for the 


6 United States. Good morning, your Honor. 


7 THE COURT: Good morning. 


8 MR. NEMTSEV: Good morning, your Honor. Max Nemtsev 


9 and Marc Fernich on behalf of Mr. Klyushin. 


09:26 10 MR. FERNICH: Good morning, Judge. 


11 NTERPRETER TETRADZE: Alexander Tetradze, Russian 


12 interpreter. 


13 NTERPRETER DORMAN: Larisa Dorfman, Russian 


14 interpreter. 


15 Interpreters duly sworn.) 


16 THE COURT: Thank you. So today we're going to hold a 


17 Daubert hearing on the statistical methodology proposed by the 


18 government. And I have a cold but it's not the big C, but 


19 didn't want to put this off, so I'm going forward with it. So 


09:27 20 I won't see you at sidebar so you can all stay healthy. 


21 So right now what I'm hoping to do is just put your 


22 expert on the stand. 


23 I do understand your expert is very ill with COVID, 


24 speaking of which, and I don't know whether you feel like you 


25 want to put him on or whether you'll just reserve on that till 
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A after the testimony. 


2 MR. NEMTSEV: think we'll see today. Depending on 
3 Mr. Clarke's testimony, we'll make a decision. 


4 THE COURT: Yes, and don't forget, all I'm determining 


5 is, A, is he qualified, B, is it relevant -—- it's easiest to do 


6 three -- and, three, is it reliable, both in terms of 


7 methodology and in terms of the data he's relying on? And I'll 


8 add a fourth thing which is to avoid the prosecutor's fallacy, 


9 which is how to state this to make sure it's not overstated and 


09:27 10 that it's clear to a jury as to whatever it is, what it means. 


11 So with that in mind, do you want to swear in the -- 


12 oh, I have two other things to say. I did get your voir dire 


13 questions. I don't know whether we'll have time today to go 


14 through them or not, but I for sure will read them and discuss 


15 with you what will go forward and what won't go forward. So 


16 right now we're still on track for that Monday, and we're 


ay pulling in a lot of jurors, so you need to let me know. 


18 Do you think -—- let me ask you this. I know you 


19 know, but I know he's got a serious case of COVID, but is 


09:28 20 definite witness? Is that something I'm going to have to 


21 about in scheduling? It's still three or four weeks away. 


22 MR. NEMTSEV: Could I let your Honor know after 


23 today's hearing? 


24 THE COURT: I can't hear a word you're saying. 


25 MR. NEMTSEV: said, can I let your Honor know after 
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ac today's hearing whether he's, you know, an absolute necessity 


2 in connection with the Daubert issues? 


3 THE COURT: I hope that he's not, like, being 


4 hospitalized or something, but you need to let me know because 


5 I'm pulling in a lot of jurors, and we're having a lot of 


6 scheduling challenges with interpreters, so it's critical. 


7 MR. NEMTSEV: will let you know, your Honor. 


8 spoke to him on Sunday, and he told me he was diagnosed with 


9 COVID and wasn't feeling well. I spoke to him on Monday, and 


09:29 10 he said he was bedridden, and I haven't been able to reach him. 
11 THE COURT: He's not in the hospital? He's not ona 


12 ventilator? 


13 MR. NEMTSEV: don't know that, your Honor. 


14 THE COURT: All right, you'll let me know. It's 


15 Obvious I shouldn't have just asked that question; we want to 


16 respect his confidentiality. All right, let's go. 
LS MR. FRANK: Thank you, your Honor. The government 


18 calls Max Clarke. 


19 MAX CLARKE 


09:29 20 having been first duly sworn, was examined and testified as 


AL follows: 


22 THE CLERK: Could you state and spell your name for 


23 the record. 


24 THE WITNESS: Max Clarke, M-a-x C-l-a-r-k-e. 


25 THE CLERK: Thank you. 


09:30 10 


09:30 20 


2] 
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MR. FRANK: May 


I proceed? 


THE COURT: Yes. 


MR. FRANK: 


IRECT EXAM 


Thank you, your Honor. 


NATION BY MR 


. FRANK: 


Good morning, Mr. Clarke. 


Good morning. 


Could you pull that microphone up close so you don't have 


to lean in every time you speak. We want to avoid neck strain? 


That's 


Is this good? 


great. Could you tell us where you work. 


I work at the Securities and Exchange Commission. 


How long have you worked at the SEC? 


What is your title? 


Senior 


I've worked there for eight and a half years. 


financial economist. 


How long have you been a senior financial economist? 


Since 2016. 


And what was your title before that? 


Financial economist, and before that economist. 


Okay. 


Beginning wi 


financial economist, what 


22 


23 


24 


25 


th your promotion from economist to 


t kinds of requirements did you need to 


satisfy to be promoted from an economist to a financial 


economist? 


A. 


need 


principles, 


to hav xpert-—level knowledge in the theory, 


and practices of financial and economic theory, as 


09:31 10 
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well as the skill to interpret complex financial 


data. 


And what do those requirements involve knowledge of? 


Working with data, finance, and statistics. 


And then ultimately you were promoted 


£ 


economist to senior 


inancial 


A. That's correct. 


Fa 


rom 


and economic 


financial 


economist? 


Okay. And that's based on seniority? 


Yes. It's three years, 


Yes. 


What division is that? 


and you become senior financial. 


Do you work within a particular division o! 


The Division of Economic and Risk Analysis. 


Yes. 


What is the Division of 


09:31 20 


21 


22 


23 


24 


25 


questions that the SEC faces. 


Q. And roughly, if you know, 


the SEC? 


Over a hundred. 


the SEC? 


Is that sometimes referred to as DERA for short? 


Economic and Risk Analysis? 


It's the part of the SEC where most of the economists sit. 


It deals with economic and statistical questions, quantitative 


how many economists are there at 


And within DERA, do you sit within a particular unit? 


Yes. 


What is that? 


09:32 10 
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I work in the Office of Litigation Economics. 


What is the Office of Litigation Economics? 


A. The Office of Litigation Economics is the part of DERA 


which deals most directly with enforcement cases. 


THE COURT: Of what? 


THE WITNESS: Enforcement cases. 


As opposed to what? 
As opposed to dealing with policy or proactive searching 


for crime. 


Q. To whom does DERA report at the SEC? Does it report to 


the Enforcement Division? 
No. 
To whom does it report? 


numbers go directly to commissioners. 


To the commissioners of the SEC? 
That's right. 


Do you have an understanding of why your division 


directly to the commissioners and not to the Office of 


Enforcement if you're working on enforcement matters? 
y 


09:32 20 


21 


22 


23 


24 


25 


I believe it's to provide more objective results. 


What do you mean by that? 


I'm able to tell Enforcement that the data does not align 


their legal theories. 


And does that happen from time to time that you tell them 


09:33 10 


09:34 20 


21 


22 


23 


24 


25 
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10 


Yes. 


Let's talk a little bit about your educational background. 


us about that. 


graduated with a BA in economics with a minor in math 
the College of William and Mary with honors. 


THE COURT: So BA in economics and -- 


THE WITNESS: A minor in mathematics. 


With honors, correct? 
That's correct. 


£ 


And as part of your studies of economics and mathematics, 


did that involve the study of statistics? 


A. 


Q. 


A. 


£ 


And was that in fact a requirement of your degree? 


It was required. I also took more classes because it's my 


personal interest. 


THE COURT: So how many statistics classes would you 


say you took? 


THE WITNESS: Probably -- I don't know off 


my head, but probably ten. 


THE COURT: As an undergrad? 


THE WITNESS: That's right. 


And tell us what subjects you studied within statistics. 


Econ stats, econometrics. I studied probability theory as 


as stochastic modeling. 


Okay let's break each of those down. What is econ 


09:34 10 


09:35 20 


21 


22 


23 


24 


25 
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regressions as it rel 


know, economic and financial data, 


economic data. 


Econometrics is the study of statistics, 


tics, economic statistics? 


What is econometrics? 


What is stochastic modeling? 


It's the study 


of random processes. 


What is probability theory? 


Those other ones are more applied. 


to do with understanding the under] 


underlying theory of statistics. 


Q. And so as part 


11 


[t's the study of statistics as it relates to economic 


particularly 
lates to large data sets that are, you 


or actually -- yeah, 


Probability theory has 


ying distribution -- 


of that study, did you study statistical 


methods, different statistical tests and the reasons that they 


work? 


Yes: 


Do you hold any certifications? 


Yes. 


What certificat 


I'm a CFA chart 


tions do you hold? 


ter holder. 


What is a CFA charter holder? 


I passed three 


exams. The exams have a body oi 


It's a chartered financial analyst. It reflects the fact 


fF knowledge 
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12 


i that test your skill in financial analysis, quantitative 


2 methods, among other things. 


3 Q. And did that also involve testing your knowledge of 


4 statistics and statistical methods? 


5 A. That's within the quantitative methods section, yes. 


6 Q. How many people who begin that process of taking those 


- three exams actually successfully complete all three? 


8 A. My understanding, it's one in five. 


9 Q. After you graduated with a degree in economics and 


09:36 10 mathematics from the College of William and Mary, where did you 


dad. work? 


12 : I worked at Cornerstone Research. 
13 : What is Cornerstone Research? 


14 ; Cornerstone Research is an economic litigation consulting 


16 : How long were you at Cornerstone Research? 
Ay . Three years. 


18 : What kinds of clients did you have when you were 


19 consulting at Cornerstone? 


09:36 20 A. Major banks, retirement plans, things like that. 


21 THE COURT: Can I back up a little. Did you get a 


22 master's at William and Mary, or is it an undergraduate degree. 
23 A. Just an undergraduate degree. 


24 THE COURT: And so did you have to write a thesis? 


25 THE WITNESS: I did not have to write a thesis, but my 
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13 


] professional —-- 


2 THE COURT: Have you written papers? 


Si THE WITNESS: I have not written any papers. 


4 ‘ What were you going to say, Mr. Clarke? 


5 : I was going to say that while I have respect for people 


6 naster's degrees, I do statistics every day. It's my job. 


7 I been my job for ten years. I'm a practitioner. 


8 ; What did your work involve at Cornerstone? 


9 . It involved summarizing large data sets and drawing 


09:37 10 conclusions from those data sets using statistics and other 
11 data techniques. 
12 Q. And what kinds of statistical methods did you use as a 


13 regular part of your work at Cornerstone? 


14 A. We used the tests that applied to the particular 


15 Situations, but we used progressions, Fisher Exact Test, among 
16 others. 
17 Q. You joined the SEC in -- 


18 THE COURT: So you yourself used the Fisher Test 


19 that's at issue here in this case? 


09:37 20 THE WITNESS: At Cornerstone, yes. 


21 THE COURT: How many times did you use it? 


22 THE WITNESS: I don't remember off the top of my head, 


23 but you only use a Fisher Exact Test under a particular 


24 circumstance. The particular circumstance is that you're 


25 comparing categorical variables. 
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14 


1 MR. FRANK: And we'll talk about that in a second. 


2 THE COURT: Okay, I'll let you go at your pace. 


i) Q. Have you also —- well, what kinds of matters have you 


4 worked on since joining the SEC in 2014? 


5 A. Sure. I work on insider trading cases, including hacked 


6 trading. I work on investment advisor abuses, as well as 


7 market manipulation and financial misreporting. 


8 Q. To what extent has your work since 2014, so the last 
9 roughly nine years at the SEC, to what extent has that work 


09:38 10 involved the regular use of statistical methods? 


11 A. I use statistics in nearly every case. 


12 Q. What kinds of statistical methods do you regularly use at 


13 the SEC? 


14 A. Again, it depends on the data at hand, but I regularly use 


LS regression analysis as well as Fisher Exact Test. 


16 ; And are there other tests that you also use? 
Ay . Sure. 


18 ; Have you won any awards for your work at the SEC? 


19 : Yes. 


09:39 20 THE COURT: Can I back up. Ina ballpark, how many 


21 would you say since you've been at the SEC, or I guess 


22 i you've been at Cornerstone, have you used the Fisher 


23 ] Test? 


24 THE WITNESS: I've used the Fisher Exact Test on the 


25 hacking cases, and I've also used --— 
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15 


i THE COURT: On the other hacking cases? 


2 THE WITNESS: Yes, that I've worked on, as well as 


3 cases that involve investment advisor abuses, so 


4 cherry-picking, for example, which is when a -—- would you like 


5 more explanation than that? 

6 THE COURT: So the ballpark, have you used it a dozen 
7 times? Have you used it a hundred times? How many times have 
8 you used it? 


9 THE WITNESS: Probably less than a dozen, but it's 


09:39 10 because a particular situation has to occur for it to happen, 


11 for it to be the correct test to use. 


12 MR. FRANK: May I proceed, your Honor? 
13 THE COURT: Sure. Thank you. 
14 ‘ Tell us about the awards you've won at the SEC. 


15 : Sure. In 2016 I won the SEC Analytic Methods Award. 


16 : And what was the Analytic Methods Award for? 
17 : That was for my work on the newswire hacking case. 


18 . In a sentence or two, tell us what the newswire hacking 


19 involved. 
09:40 20 A. The allegations were that hackers got access to press 


21 releases through hacking newswires like Business Wire and PR 


22 Newswire before they were released to the public, and traders 


23 were trading on this inside information prior to the public 


24 dissemination of news. 


25 Q. So in contrast to this case where the allegation is that 


09:40 10 


09:41 20 


21 


22 


23 


24 


25 
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there were SEC filings that were accessed before they were 


released to the public, in that case it was press releases that 


were allegedly accessed before they were released to the 
public? 


Ps Yes. 


O. And that was from the PR Newswire and Business Wire 
databases? 


A. 


Q. And those press releases concerned what? 


A. Earnings. 
Q. And what analytical methods did you employ in your work in 


that case? 


A. We used Fisher Exact Test and permutation test. 
And you won an award for that from the SEC? 
That's correct. 
And in that case, was that a civil case, a criminal case, 


or both? 


A. Both. 


How was your analysis used in that case? 


supported an expert who presented these to a jury. 


So you didn't personally testify before the jury? 


That's correct. 


But your analysis was provided through the expert 


testimony to the jury? 


A. Yes. 
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i Q. What other awards have you won since being at the SEC? 


2 A. Forgive me if I forget the dates, but I think in 2019 


3 won the DERA Director Award, and in 2021 I won the Matthews 


4 Award. 


5 ‘ Okay, let's take each of those in turn. What did you win 


6 DERA Director's Award for? 


i A. Sure. So one part of my job that's not related to 


8 enforcement has to do with calculating the fee rate that the 


9 SEC charges to all stock transactions. 


09:42 10 : Why does the SEC charge a fee to all stock transactions? 


11 A. The SEC is a self-funded agency, so there's an 


12 appropriation from Congress that's for, you know, about 


13 $2 billion. The SEC raises that money through a transaction 


14 fee that's charged to every single stock trade that's made, and 


LS they take -- you have to predict how much stocks will trade, 


16 and then you divide that by the $2 billion appropriations to 


17 arrive at a fee rate. I think last year was $22. But I use 


18 statistics, statistical method and statistics to come up with 


19 that number. 
09:42 20 : And how many economists did you say there were at the SEC? 


AL F Over a hundred. 


22 : How many economists were selected to calculate the fee 
23 
24 


25 : And then you won an award for that? 
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1. A. That's right. 
2 Q. Tell us about the third award that you mentioned, the 
3 Matthews Award? 


4 A. That was related to the EDGAR hack. 


5 QO. What was the EDGAR hacking case about? 


6 A. EDGAR hacking case had to do with an allegation that 


7 there was unauthorized access onto the EDGAR, which is the 


8 SEC's own system for disseminating news. There's something 


9 called "test filings" that the hackers had access to prior to 


09:43 10 the public dissemination of the news. 


11 Q. What kind of filings were alleged to have been hacked {| 


12 the SEC's EDGAR database in that case? 


13 A. Earnings, earnings. 


14 ‘ And what did your work in that case involve? 


15 : I summarized data and drew conclusions using statistics. 


16 tatistics I used were the Fisher Exact Test. Yeah, 


17 t use the permutation test in that one. 


18 THE COURT: I lost you there. 


19 THE WITNESS: Sorry. 


09:43 20 THE COURT: So you used the Fisher Exact Test, and 


21 then what did you say? 


22 THE WITNESS: I used two tests in this case. I used 


23 the Fisher Exact Test and the permutation test. I'm saying, in 


24 the EDGAR hack, I used the Fisher Exact Test but not the 


25 permutation test. 


09:44 10 


09:44 20 


21 


22 


23 


24 


25 
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in those cases that we've talked about, did you 


you ever testified in a court proceeding? 


How often? 


[ testified one time. 


And what case was that? 


I believe it was called SEC v. Sargent. 


And was that in this courthouse? 
Yes, it was. 


Who was the presiding judge in that case? 


[ think his name was Young. 

Judge Young? 

Yes. 

And did you testify in that case as a fact witness or as 


an expert witness? 


A. I believe as a fact witness. 


Ox And what was the subject of your testimony in that case? 


A. It had to do with a —- there's a Supreme Court case called 


Liu that has to do with giving money back to people who were 


harmed. The question was, was it feasible to calculate who was 


harmed in this case, and I testified that, yes, it was 
feasible. 


Or And do you know what the Court found? 
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1 : They found that it was feasible. 


2 : And that was a bench trial or a jury trial? 


3 ; Bench trial or -—- yes. 
4 THE COURT: What was it, a restitution issue? Was it 
5 a criminal -—- 


6 THE WITNESS: It was criminal -- or I don't know. 


7 MR. FRANK: It was an SEC case, your Honor. It was a 
8 Civil case. 
9 THE COURT: Civil, okay. 


09:45 10 . And did that involve the use of statistics? 


11 . Did the ultimate conclusion? 


12 ‘ Yes, involve the use of statistics in that case? 
13 : No. 


14 : Have you submitted a written expert report in connection 


15 any other cases? 
16 : Yes. 


17 ‘ Were those civil -- well, which case was that? 


18 : U.S. v. Soady. That was in the Northern Dis 


19 Alabama. 
09:46 20 : That was a criminal case? 
21 


22 Q. And in that case, did your work involve the use of 


23 statistics? 
24 A. 


25 Q. What was your work in that case? 
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21 


a A. It was for a sentencing hearing. They wanted me to 


2 calculate damages on a pump-and-dump scheme. 
3 Q. And what statistical methods did you use in that case, if 


4 you recall? 


5 : I used regressions. 
6 ; And how was your work used in that case? 


i : I believe it was accepted by both sides, and they accepted 


8 my numbers to use in sentencing. 


9 Q. And that was submitted as expert testimony or an expert 


09:46 10 report? 


11 A. Yeah, I wrote an expert report, yes. 


12 Q. As a general matter, Mr. Clarke, in your work at the SEC, 


13 do you use a variety of statistical methods? 


14 A. I use the statistical tests that are relevant to the 


15 dataset at hand. It depends on the facts and circumstances of 
16 the case. 


ay Q. What do you mean when you say that? 


18 A. I mean that different data —- the way the data is 


19 constructed can lead to the use of differen tistical 
09:47 20 tests —- 
21 : And so -- 
22 : —- to being the best test. 
23 Q. And so how do you decide which is the best test in a given 


24 circumstance? 


25 A. Uhm, I use my knowledge of statistics. I consult with 


09:47 10 
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other experts at the SEC, and then I make a decision. 


Q. And so in your work at the SEC, you use tests like the 
Fisher Exact Test? 

That's correct. 

You use regression analysis? 


I do. 


What are some of the other tests you've used? 


I've used T-tests. I've used ANOVA tests. I've used 


different types of regressions, like logistical regression. 


Ou Let me ask you now specifically about the Fisher Exact 


Test. You mentioned a moment ago in answer to Judge Saris' 


question that it involves categorical variables. 


A. 


Q. What are you testing in the Fisher Exact Test? 


A. You're testing the independence of two categorical 


variables. 


Q. And what are categorical variables? 
A. They're variables that you can group into categories, 


is it red or is it blue? 


09:48 20 


21 


22 


23 


24 


25 


£ 


Q. As distinguished from what? 


£ 


distinguished from a continuous variable, some 


that, like, you know, stock prices over time move a lot. 


Q. So time, would time be a categorical variable or a 


continuous variable? 


A. A continuous variable. 


22 
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1 O;. Would the question whether somebody traded or not be a 


2 categorical variable or a continuous variable? 
3 A. Sorry. Could you repeat the question. 


4 Q. Whether or not somebody traded, would that be a 


5 categorical variable or a continuous variable? 


6 A. That would be a categorical variable. 


7 Q. So, again, in what circumstances do you use the Fisher 


8 Exact Test? 


9 A. When you're comparing two categorical variables. 


09:49 10 Q. It's been suggested in some of the defense filings in this 


11 case that the Fisher Exact Test is poorly suited for situations 


12 where the data comes from historical events rather than a 


13 controlled experiment. Do you agree with that statement? 


15 , Why? 


16 A. The study of finance almost always uses historical data. 


17 So anytime a Fisher Exact Test is used in the context of 


18 finance, it's on historical data. 


19 Q. And is the Fisher Exact Test, in your experience, used in 


09:49 20 finance? 
21 : Yes. 
22 : Have you used it? 
23 : Yes. 
24 : Are you aware of other experts using it? 


25 . Yes. 
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24 


1 ©. It's also been suggested by the defense that the Fisher 


2 Exact Test involves simplistic assumptions that lend themselves 


i) to extreme results because the test is comparing two categorical 


4 variables and not accounting for other factors. Do you agree 
5 with that statement? 
6 ; No. 


. - Why not? 


8 ‘ I think that the Fisher Exact Test does what it says it 


9 does: It's comparing two categorical variables. So it's 


09:50 10 answering a question, which is, are these two variables 


ala correlated or not? If you want to do something different, 


12 you'll be answering a different question. 


13 Q. It's been suggested that the test -- 


14 THE COURT: Can I back up. Tell me about this test. 
LS How long has it been a test that's been used by statisticians? 


16 THE WITNESS: The Fisher Exact Test? 


LS) THE COURT: Yes. 


18 THE WITNESS: It is a —- like, Fisher was one of the, 


19 you know, like, original statisticians when the field was 
09:50 20 coming up, so it's been around for a really long time. 
21 THE COURT: And is it something you studied in school? 


22 THE WITNESS: Yes. 


23 THE COURT: And is your work reviewed at the SEC to 
24 make sure you're using the right test? 


25 THE WITNESS: Yes. 
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ac QO. And in fact, Mr. Clarke, in this case, was your work 


2 reviewed before it was submitted? 


3 < Yes. 


4 : So it's checked by other financial economists? 


3 ‘ That's correct. 


6 ‘ And you've also checked the work of other financial 


7 economists at the SEC, correct? 


8 ‘ That's right. It's standard practice. 


9 : The suggestion has been made that the Fisher Exact Test 


09:51 10 ves simplistic assumptions that lend themselves to extreme 


1] ts when there's a large dataset. Do you agree with that? 


12 - No. 


13 ‘ Why not? 


14 : In statistics, it's usually better to have -- you can make 
15 better conclusions when you have more data. 
16 Q. So let me ask you about how you used the test in this 


17 particular case. What questions were you seeking to answer 


18 when you employed the Fisher Exact Test here? 


19 A. Is there a relationship between the traders' trading and 


09:52 20 whether or not it was a DFIN or Toppan earnings event? 


21 THE COURT: Say that again. Whether there's a 


22 correlation between what? 


23 THE WITNESS: If the traders traded it, and if it was 


24 IN or Toppan earnings event or not. 


25 z So on the one hand, the categorical variable was, did the 
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] traders trade in an earnings event? 


2 A. That's right. 


3 O. And on the other hand, the categorical variable was, was 


4 that earnings event handled by DFIN or Toppan Merrill as the 
5 filing agent? 


6 THE COURT: You know, TI don't know the words you're 


8 MR. FRANK: Let me define the companies. 


9 ©. I'm using the terms DFIN and Toppan Merrill. DFI] 


09:52 10 abbreviation for a company called Donnelly Financial? 
11 A. That's right. 


12 : Toppan Merrill is another company? 


13 . That's right. I think those are the at-issue filing 


14 agents in this case. 


15 QO. And so those are the 


16 THE COURT: So those are Financial Analysts 1 and 2? 


LS) THE WITNESS: Yes. 


18 THE COURT: We haven't been using those terms, so 


19 excuse me for my -—- 


09:53 20 MR. FRANK: I apologize, your Honor. That's my fault. 


21 And so Toppan Merrill, just for reference, is Filing Agent 1, 


22 and Donnelly Financial, which is abbreviated DFIN, is Filing 


23 Agent 2. And Toppan Merrill is often abbreviated TM. 


24 Q. Okay, so just to step back, the first question you asked 


25 was, did the defendant trade around a particular earnings 
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ac event, categorical variable, versus was that earnings release 


2 filed with the SEC's EDGAR database by Donnelly Financial or 
3 Toppan Merrill? 
4 A. That's correct. 


5 THE COURT: All right, so let me ask you this: These 


6 two firms, how many of these -- what are they, financial —- 


7 what do you call them? 


8 THE WITNESS: Filing agents. 


9 THE COURT: Filing agents, how many of them are there? 


09:54 10 THE WITNESS: There are a bunch, but there are three 


12 COURT: Three big ones? 


13 WITNESS: Yes. 


14 COURT: Who's the third one? 


iS) THE WITNESS: Workiva, W-o-r-k-i-v-a, Workiva. 


16 THE COURT: And are there specialties? In other 


Ly words, does Workiva, f I’m pronouncing it correctly, 


18 specialize in a special kind of security? 


19 THE WITNESS: No. They all span kind of all 


09:54 20 securities. 


21 THE COURT: So it's not that one does more S&P 500 


22 than another? 


23 THE WITNESS: I don't think that's the case. 


24 MR. FRANK: We're going to talk about that in detail, 


25 your Honor. 
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i THE COURT: Okay, but I'm just trying to understand. 


2 So when you're doing this analysis, can you explain it by the 


3 fact that some financial agents do one kind of stock versus 


4 another kind of stock? 


5 THE WITNESS: No. 


6 THE COURT: Why? 


7 THE WITNESS: So if you look at the filings and you 


8 split it up by SEC code, which is the sector, you can see that 


£ 


9 the market share is pretty similar for across all sectors of 


09:55 10 Donnelly Financial and Toppan Merrill. So the 44 percent, it 


‘lal might be like 48 percent or something for the agricultural 


12 sector or something, and then it's 41 percent for the 


13 manufacturing sector. Does that make sense? 


14 MR. FRANK: I think it will make more sense when we 


15 get to => 


16 THE COURT: All right, I'll let you keep going because 


17 you're probably anticipating my questions. Go ahead. 


18 MR. FRANK: Thank you, your Honor. 


19 Q. So that was one question that you asked, and then what was 
09:55 20 the second question that you asked? 


21 A. There's something that's in finance that's called an 


22 "earnings beat" and an "earnings miss," and that's when the 


23 company does much better than what the market expects, and the 


24 stock prices, you know, you would expect to go up. And the 


25 other way is that there's an earnings miss where the earnings 
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1 is way worse than people expected. So I looked at earnings 


2 beats and earnings misses as one of my categorical variables, 


Si and then traded long or short for my other one. 


4 ‘ Okay, so just to break that down, and we're going to do 


5 | in detail, but on the one hand, the categorical variable 


6 | you examined was, did the trader trade long; that is to 


7 id they buy stock? 


8 : ight, did they expect the stock price to go up. 


9 : I uying stock? 


09:56 10 : By buying stock. 

11 Os Or did they sell the stock short; that is, make a bet that 
12 the stock was going to go down? 

13 A. Yes. 


14 Q. And on the other hand, the categorical variable was, did 


15 the company exceed expectations; that is, beat earnings 
16 expectations? 


Ly 7 Yes. 


18 : Or did the company fall short of earnings expectations? 


19 : That's right. 


09:56 20 : So those were the two categorical variables for that test? 


21 : That's right. 


22 : Okay. And why did you use the Fisher Exact Test in those 


23 two instances? 


24 ‘ Because I wasS comparing two categorical variables. 


25 : Okay, so now I want to try to answer the Judge's questions 
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about the first test 


traders trade around 


09:57 10 


09:58 20 


21 


22 


23 


24 


25 


that you did, which is to say, did the 


an earnings event, and was that earnings 


filing filed by one of these two filing agents? 


first of all 


, let's talk about what data you used. 


we could call up Exh 


ibit A, and if you could tell the Court, 


what is Exhibit A? 


A. I don't see any 


THE COURT: 


THE CLERK: 


Do you see it on your screen? 


thing on my screen. 


Is your screen not working? 


You might have to flip the other screen 


over. This always happens to the screen. It was working this 


morning. You can tap it. 


THE WITNESS 


MR. KOSTO: 


: It looks good. 


Your Honor, if I could approach the 


witness, I could give him a paper copy to work with that 


matches what's on the Court's screen. 


THE COURT: 

MR. KOSTO: 

THE COURT: 
Q. Are you looking 
and Traders"? 


A. 


Did you prepare 


did. 


What is it? 


Yes. 


So may I] 


Yes. 


at Exhibit A, "Trading Blotters: Accounts 


that document? 
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1 A. This shows the trading blotters that I analyzed in this 


2 case. It shows that there are 22 accounts across six brokers 
3 and eight traders. 


4 OQ. Okay. So what's a trading blotter? 


5 A. A trading blotter is when you request data from a company 


6 and you want to see their trades. It has all the buys and 


7 sells, prices and quantities that the traders traded; and the 


8 ticker symbol is some identifier of what they traded in. 


9 : So it's a record of their trading for those accounts? 


09:59 10 : That's right. 


11 . Okay. And how many different traders are identified here? 


12 ; Bight. 


13 i Okay. And how many different brokerage firms were 


14 involved? 


15 A. Six. 


16 Q. And how many different accounts at those brokerage 
17 did you examine? 


18 ; Twenty-two. 


19 : And what period of time did your analysis focus on? 


09:59 20 : January 1, 2018, through September 30, 2020. 


21 : And why that time period? 


22 : I was asked to look at that time period. 


23 ; And you're aware that that's the time period of the 


24 alleged conspiracy in this case? 


25 A. Yes. 
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1 j Okay. And why these traders and these accounts? 


2 : These were the traders and accounts I was given. 

3 ‘ So this was the trading dataset that you started with? 

4 ‘ That's right. 

5 : When you looked at this set of trading data, were you able 


6 to determine whether these eight traders, the defendant and the 


7 other traders, traded exclusively around earnings announcements, 


8 or did they trade at other times as well? 


9 A. They traded at other times as well. 


10:00 10 Q. Could we look at Exhibit -- 


11 MR. FRANK: And I don't know if your Honor would like 


12 me to offer these exhibits formally or not, and so I offer 


13 Exhibit A. 


14 THE COURT: All right, I allow that. 


15 (Exhibit A received in evidence.) 


16 Q. Could we take a look now at Exhibit B. Did you prepare 


LS) Exhibit B, Mr. Clarke? 


18 A. I did. 


19 F What is Exhibit B? 


10:00 20 : It shows the summaries of the profits that they made for 


21 1 their trading for all accounts and the different way they 


22 lit it up. 


23 MR. FRANK: So the government offers Exhibit B. 


24 (Exhibit B received in evidence.) 


25 : for purposes of preparing the summary in Exhibit B, did 
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1 you look at all of those accounts that we looked at in 


2 Exhibit A? 


3 A. Yes. 


4 Q. Okay. And just at a high level, tell us, what did you 


5 determine that is se Forth in Exhibit B? 


6 A. Sure. So, first, I split the data between long term and 


a short term. So I define short term as three days or less, 


8 trading that opens and closes within three days, and that's a 


9 common heuristic. And then I looked at trading that was around 


10:01 10 earnings events. That means a trade that was opened before an 


11 earnings was released and closed after that it was released. 


12 Q. Okay, so let's break that down. So you distinguished 


13 between long-term trading events and short-term trading events? 
14 A. That's right. 


15 Q. And you defined short-term trading events as where the 


16 trade was open -—- that is, the stock was bought or sold on day 
17 one —- and then unwound —- that is, sold or bought -- within 


18 three days? 


19 ; Yeah, that's right. 


10:02 20 ; And long-term events were longer than that? 
21 : That's right. 


22 : And why did you define short-term events that way? 


23 : When I looked at the data, most of the trading occurring 


24 in short-term trades, three days or less. 


25 Q. And then you said you distinguished between non-earnings 


10:03 10 


10:03 20 


21 


22 


23 


24 


25 
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A. So earnings are required to be disclosed by companies, and 


so there's a particular type of trading stra 


CEGQY 


that's called 


"earnings trading." It's a very common stra 


Legy 


do. And what that entails is that you expect the 


move on these earnings announcements, so you 


that people 


market to 


open 


earnings, and then you close after. I saw in the 


they do this a lot. 


Q. Okay. And so how did you define an earnings 


earnings trade, for purposes of this analysis? 


before th 


data that 


A. I said if the trade was open within three days of the 


earnings announcement and closed after. 


Q. And how often are companies in the United States required 


to report earnings? 
Quarterly. 
So every three months? 


That's right. 


And looking at your chart, what did you determine, when 


you looked at all of the trading by those eight traders during 


that time period, about the concentrations of their trading? 


A. Could you repeat the question? 


Q. Maybe it was a bad question. What did you determine they 


did most often during that period? 
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i THE COURT: Did your screens just go off? Mine has. 


2 THE WITNESS: My screen just came on. 


Si THE COURT: You took mine. I don't have it. Anyway, 


4 I have another one here. OkKay. 


5 QO. So taking a look at Exhibit B, what did you determine 


6 about the frequency of their trading in these different styles? 


7 Aus Sure. The vast majority of their profits is coming from 


8 earnings trades, and the, you know, significant majority of 


9 their trading was in earnings trades. 


10:04 10 Q. Okay. So it looks like you've identified a total of 
11 under 3,000 total trading episodes; is that right? 

12 A. That's right. 

13 Q. And what's a trading episode? 

14 A. That's one instance of a position being opened and a 
15 position being closed. 

16 Q. Okay. So that's a roundtrip? 


17 A. A roundtrip trade, sure. 


18 Q. Okay. And of those 2,900 trading episodes, how many 


19 earnings-related? 


10:04 20 3 That would be 1,988 plus 50, so that's —-- 


21 - About two-thirds? 


22 : About two-thirds, yes. 
23 : Okay. And of those earnings episodes, those trading 
24 episodes around earnings -—- and we'll talk about how you 


25 determined this in a minute —- but were you able to determine 
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1 how many of those earnings filings were handled by Donnelly 


2 Financial or Toppan Merrill? 


3 : Yeah. I think it's something like 98 percent. 
4 ‘ Okay. And that's the 1,988 number that we're looking at? 


5 : That's right. 


6 : And let's look at profitability. In this period of 


7 roughly two years and nine months, how much did the eight 


8 traders earn from their earnings trading around events handled 


9 by Donnelly Financial or Toppan Merrill? 


10:05 10 A. So) ,.o million, 

11 Ox Okay. How much did they earn from their earnings trading 
12 in companies whose earnings filings were handled by other 

13 companies? 

14 A. $544,000. 

15 Q. Okay. And what about relative to all their other trading, 


16 how much was their earnings trading in Toppan Merrill and 


17 Donnelly Financial companies versus all their other trading? 


18 : Significantly larger. 


19 : Okay. So overall they made over this period $89 million? 
10:06 20 : That's right. 


21 : And is it fair to say, they lost money collectively on all 


22 their other trades but made $97 million on the earnings trading 
23 around Donnelly and Toppan? 
24 A. That's right. 


25 Q. And then on the far right, you have "Return per episode." 
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al What is that? 


2 A. That's per episode, the return on a single trade, the 


3 average return, money-weighted average return. 
4 Q. So just in one trading episode, it could be a few days; it 


5 could be longer than that? 


7 . So what does this column show? 


8 A. The return was the highest on the DFIN/Toppan Merrill 


9 earnings trades. 


10:07 10 Q. Double what it was on other earnings trades? 


11 THE COURT: Where do you show that? I'm sorry. 


12 MR. FRANK: On the far right, your Honor. 


13 THE COURT: Per episode, and then where does it say 


14 that it's making more money? 


15 MR. FRANK: So if your Honor looks at the very bottom, 


16 you'll see the return per episode for trades related to DFIN 


LS and Toppan Merrill clients. That's 4.4 percent. 


18 THE COURT: Where are the others? Oh, others 2.2, so 


19 i double? Is that it? 


10:07 20 MR. FRANK: Correct. 


21 THE COURT: Let me ask you this: What percentage does 


22 third filing agent have? 


23 MR. FRANK: We're about to look at that, your Honor. 


24 : Could we look at Exhibit C. Did you prepare Exhibit C? 


25 . I did. 
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al: : What is Exhibit C? 


2 : Exhibit C is just graphing the previous exhibit's profits. 


3 : Okay. So this is just showing that they made close to 


4 $100 million trading in the earnings of DFIN and Toppan Merrill 


5 clients, and they made, it's fair to say, de minimis amounts or 


6 lost money on all their other trading? 
7 A. Much smaller amounts, yes. 


8 MR. FRANK: The government offers Exhibit C. 


9 (Exhibit C received in evidence.) 


10:08 10 Ox And now I'd like to take a look at exhibit D. 


1] Exhibit D? 


12 A. Exhibit D shows the amount invested with outside money 


13 versus the amount of money they made from this DFIN/Toppan 


14 Merrill earnings trades. 


15 Q. Okay. So here we're just looking at their trading in 


16 and Toppan Merrill clients, just those earnings trades? 
17 A. Yes. 


18 Q. Okay. And directing your attention to the second line for 


19 Mr. Klyushin, what do you mean by "maximum net deposit"? 


10:08 20 A. So I reviewed cash flow files and account statements, and 


21 I could see when he deposited money into his account, and this 


22 was the maximum amount that he deposited from using outside 


23 funds. So sometimes he transferred money between accounts. I 


24 idn't want to include that because that would be money from 


25 within the strategy. So it was money coming from outside 
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il investing into the strategy. 


2 Q. So is it fair to say that the total amount of outside 


3 money he invested in his account, his brokerage account, during 


4 this time period was $2.1 million? 


6 ‘ during that same period, how much did he personally 


7 generate from trading around the earnings of clients whose 


8 earnings were filed by Donnelly Financial or Toppan Merrill? 


9 . About $21 million. 


10:09 10 : And what was his return on that portion of his trading? 
11 : 895 percent. 
12 : So almost 900 percent? 


13 : That's right. 


14 : Now, we see different return rates for the different 


15 traders. Can you tell us why that is? 


16 : It has to do with how long they're in the strategy, how 


17 trading episodes they traded, and the amount that they 


18 ted per trade. 


19 . So some of these traders started earlier? 

10:10 20 : That's right. 
21 : And they bet different amounts on the different trades? 
22 : That's right. 


23 MR. FRANK: The government offers Exhibit D. 


24 Exhibit D received in evidence.) 


25 : I'd like to move to Exhibit E. Okay, and I 
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i this will now address Judge Saris' question of a moment ago. 


2 Did you prepare this chart? 


3 F Yes, I did. 


4 ‘ What does it show? 


bs F It shows t in the dataset that I looked at, I observed 


6 that 96 percent f Klyushin's earnings trades were in DFIN and 


. Toppan Merrill. I thought that needed some context. Well, 


8 what does the market share look like? And I found that to be 


9 44 percent, so -—- 


10:10 10 Q. Let's talk about how you determined the market share 


11 attributable to DFIN and Toppan Merrill. So what did you do to 


12 figure out what their share of the market was? 


13 A. Sure. I used the accession number of the earnings filing. 


14 Q. What is the accession number? 


15 A. Sure. At the SEC, every time an earnings file is 


16 submitted to the EDGAR system, an accession number is 


17 generated. The accession number tells us who filed. 


18 Q. Okay. So for every single earnings filing, you can 


19 determine who filed it? 


10:11 20 A. That's right. 


21 : By accession number? 


22 ‘ Yes. Well, some of them are self, but, yes. 


23 ; Okay, so we'll talk about that in a minute, but there is 


24 an accession number associated with every filing? 


25 A. That's right. 
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al OQ. So during this period, January 1, 2018, to September 30, 


2 2020, what did you do with respect to the earnings filings that 


3 were filed during that period? 


4 A. Can you repeat the question. 


5 O- What did you examine with respect to the earnings 


6 1 the earnings filings that were filed between January 1 


7 18, and September 30, 2020? 


8 < I looked at the earnings announcements, and I looked at 


9 the filing associated with it, and I looked at the accession 


10:12 10 number. If the accession number said that it was DFIN or 


11 Toppan Merrill, I said it was DFIN or Toppan Merrill, and I 


12 just summarizing that data. 


13 Q. So when you summarize that data, the data that you're 


14 looking at is every single earnings filing with the SEC during 


15 that period, correct? 
16 A. For US exchange-listed companies. 


17 Q. So for US exchange-listed companies, you looked at every 


18 single earnings filing with the SEC from January 1, 2018, to 


19 September 30, 2020? 


10:12 20 A. 


21 Q. And you examined the accession number with each and every 


22 one of those filings? 


23 A. That's right. 


24 Q. And you found that 44 percent were filed by DFIN or Toppan 


25 Merrill? 
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2 ‘ What is the other 56 percent? 


i A. The other 56 percent includes accession numbers that are 


4 associated with other filing agents, as well as accession 


5 numbers that are associated -—- the CIK is the same as the 


6 company, so it's self, self files. 


7 Q. So what does it mean to be self-filed? 


8 A. The way I think about it is, you can use H & R Block to 


£ 


9 have someone file something for you, your taxes for you, or you 


10:13 10 can use TurboTax and you can file it by yourself. 


‘lal I think the filing agents to the SEC, it's a similar 


12 thing. So the self-filers are people that use software, and 


13 then that software -- submits it to the SEC, and it comes into 


14 the SEC as an accession number of self. 

15 : So a company could use software and file its own earnings? 
16 ; Yes: 

17 : That would put it in the 56 percent? 


18 : That's right. 


19 : Okay. Or it could hire somebody other than Donnelly 


10:14 20 Financial or Toppan Merrill to file that filing? 


21 : That's right. 
22 : That would also put it in the 56 percent? 


23 : Correct. 


24 : it could hire Donnelly Financial or Toppan Merrill and 


25 file its earnings? 


10:14 10 


10:15 20 


21 


22 


23 


24 


25 
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Use them to file on its behalf. 


Yes, to file on its behalf. 


And that would put them in the 44 percent? 
That's right. 


Why did you use this method to analyze the company's 


market share versus their just their client list? 


43 


A. Because what we're concerned about is the literal earnings 


release, and this is a measure of the earnings releases. 
Q. So which is a more accurate measure, in your view? 
A. This, using the accession number. 

Q. And explain to us why the client list would not be 
accurate. 


A. A client list would tell you if it was ever a client. 


Clients can change customer. You can use DFIN/Toppan for one 


earnings, and you could use Workiva for another. 


Q. So that's the 44 percent. What is the 96 percent? 


A. I use the same exact method on his -- I'm sorry -- on 


Klyushin's trading. 


Q. And what did you observe about Mr. Klyushin's trading? 


A. Using accession number, you find 96 percent of the 


earnings trades he traded on were associated with DFIN and 
Toppan. 
Q. Once you had done this analysis and seen this disparity, 


what did you do next? 
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al : I saw this phenomenon, I saw this pattern in the data, and 


2 I thought it made sense to run a statistic to see if it was 


3 relevant, if it was statistically significant. 


4 p If what was statistically significant? 


5 ‘ This difference. 


6 ‘ And so what was the test that you ran? 


i : I ran a Fisher Exact Test. 
8 : And, again, why based on this data did you choose to run 


9 test? 


10:16 10 : Because I was comparing two categorical variables. 


11 Q. And, again, the categorical variables were on the one hand 


12 what? 


13 A. Traded verse not traded and filed by DFIN/Toppan or not 


14 filed by DFIN/Toppan. 


15 : Are you familiar with the term "null hypothesis"? 


16 : Yes. 


17 : What's a null hypothesis? 


18 : It's the thing you test in statistics. The null 


19 thesis here is that they're uncorrelated; there's no 


10:16 20 tionship. 


21 Q. So the thing you were testing is that there was no 


22 relationship between how he traded and who the filing agent 


23 was? 


24 A. That's right. 


25 Q. Can we look at Exhibit F, please. Did you prepare 
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1 Exhibit F? 


2 : Yes. 


3 ‘ What is it? 


4 z It's similar to the previous exhibit, but it breaks it out 
5 by the eight traders. 


6 MR. FRANK: Okay, so the government offers Exhibit F. 


7 I'm not sure I offered E, so I offer that as well. 


mu 


8 (Exhibits E and F received in evidence.) 


9 Q. And let's take this now column by column. In the second 


10:17 10 column, after the traders' names, it says "IBES earnings events 


11 during the traders' trading period." What is an IBES earnings 


12 event? 


13 A. BES is the Institutional Brokers' Estimate System. It's 


14 a common dataset to aggregate earnings announcements. I looked 


15 for the total number of earnings announcements in that dataset 


16 for U.S. exchange-listed companies for this time period. 


ay Q. Okay, so that's all of the earnings announcements during 


£ 


18 the period in which each of these individuals traded between 


19 January 1, 2018, and September 30, 2020? 


10:18 20 A. Yes. Or, sorry. It depends on -—- yes, during the time 
21 period in which the trader trades. That's why that number 


22 changes. 


23 Q. So explain that. If it's all for the same period between 


24 January and September, '18 to '20, why are the numbers 


25 different for each trader? 


10:19 10 


10:19 20 


21 


22 


23 


24 


25 
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A. I compared the number of earnings events during the time 


period in which the traders traded to the number of earnings 


events that were issued during that time period. 


the most because he traded the 


January 1? 
That's right. 
He started later? 


That's correct. 


So he has fewer earnings events during the period of 


personal trading? 


A. That's right. 


So Sladko 


longest, and M13 has the 


t because he traded the shortest time period. 


V 


see. So Mr. Klyushin didn't actually start trading on 


Q. Okay. So in his case, it's 38,359 earnings announcements 


during that period? 
That's right. 


In all U.S. listed stocks? 


Correct. 


Okay, what's Column 2? 


Column 2 is, using that accession number, 


Q. Okay. So during the period that Mr. 


those 38,000 earnings events, 16,8 


A. That's right. 


Donnelly Financial or Toppan Merril 


IN and Toppan Merrill earnings events of the 38,000? 


Klyushin traded, 


10 were actual] 


1? 


ly filed by 


which ones were 


of 


10:20 10 
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And you did that on a release-by-release basis? 


That's correct. 


That's the 16,000 


Okay. 


So that is 


A. 
Q. Now, 


events in 


In Column 3, what is the 44 percent? 
divided by the 38,000. 
the market share that you attributed to 


Donnelly Financial and Toppan Merrill? 


you have a slightly different number of earnings 


Column 1 


number of 


Column 2. 


in Column 


10:20 20 


21 


22 


23 


24 


25 


Donne] 


ly Financial 


for each of these traders and a different 


and Toppan Merrill events in 


Why is the 44 percent the same all across the board 


3? 


Because the market share of DFIN and Toppan doesn't change 


time by more than 1 percent. 


oF So even though you were examining it on an 


earnings-—release-by-—earnings-release basis, you found that the 


number stayed consistent? 


That's right. 


What is Column 4? 


That's the number of events traded by the trader. 


So that's the number of earnings announcements around 


which they actually traded? 


A. That's correct. 


Q. Okay. 


A. 35.6. 


And what kinds of 


fF that number for Mr. Klyushin? 
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al: : And what is Column 5? 


2 : That's the number of, of those 356, how many were 


3 DFIN or Toppan Merrill. 


4 ‘ And what is the number for Mr. Klyushin? 


5 , 343. 


6 . And what does the next column show? 


7 : The percentage, 343 of 356, and that's 96 percent. 


8 : So that's the 96 percent that we looked at on the bar 
9 chart that we looked at a moment ago? 
10:21 10 A. Yes. 


11 Ox Okay. And how does it compare, that percentage of trades 


12 around DFIN or Toppan Merrill events, how does that compare 


13 across the eight traders? 


14 : It's similar. 

15 : So at the low end, it's 96 percent? 
16 : That's right. 

17 : And at the high end? 


18 : 99 percent. 


19 ‘ Okay. What does the second-to-last column reflect? 


10:21 20 : This is the expected it's the amount that he traded 


21 times the 44 percent. So that's the expected number that you 


22 would -- it's the number of DFIN and Toppan null events you 


23 would expect under the null hypothesis. 


24 Q. Okay. So if his trading had reflected the 44 percent of 


25 the market that they had, it would have been 166 trading events 
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A instead of 343 trading events? 


2 A. That would be your expectation if they're unrelated. 


3 Q. Let's look at the last value, p-value. What is that? 


4 A. That's the probability of seeing this pattern in the data 


5 if you make the assumption that it was random. 


6 Q. So, in other words, your null hypothesis again was what? 


7 A. My null hypothesis is that the relationship between DFIN 


8 and Toppan and their trading was uncorrelated, unrelated. 


9 Q. And what does the p-value reflect? 


10:22 10 A. The p-value reflects the probability of seeing this 


11 96 percent or something more extreme if you assume that it's 


12 uncorrelated. 


13 Q. Okay. So if there's no relationship between the traders 


14 trading and who the filing agent is, if that is not correlated, 


15 the p-value tells you how often you would see what you actually 
16 saw in the data? 


17 A. That's right. 


18 Q. Okay, which is that he traded in 343 DFIN and Toppan 


19 Merrill events out of 356 events? 

10:23 20 : Yes. 
21 : And what was the p-value in this case? 
22 : Less than one in a trillion. 
23 : That's less than one in a trillion? 


24 : That's correct. 


25 ; So just so I understand, that means that if there's no 
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] relationship between his trading and who the filing agent is, 


2 you would expect to observe this result less than one in a 
3 trillion times? 
4 A. That's right. 


5 0. Can you explain that math. 


6 A. Sure. What the Fisher Exact Test does in this case is, 


7 the same as the setup of a classic probability question, which 


8 is, if you have a jar that has 38,000 marbles in it and, you 


9 know, 44 percent are red, what's the probability of, after 


10:24 10 drawing 356 marbles, that 343 are red? 
11 Q. So if you have a jar with 38,000 -- 


12 THE COURT: So the math is what? Is it just 


13 continuing to multiply one out of two, one out of two? 


14 THE WITNESS: Yes, that's right. It's similar to, you 


15 know, if you say 44 percent is 50 percent, you say how probable 
16 is it to flip a coin 356 times and get heads 343? 
17 THE COURT: And the answer is —- it sounds very 


18 basic -—- you're just multiplying? 


19 THE WITNESS: Yes. It's a little more complicated, 


10:24 20 but, yeah, it's kind of -—- the reason why the number gets so 


21 big is because it's one over two to the 343th or something. 


22 It's really -- 


23 THE COURT: It's exponential? 


24 THE WITNESS: It's exponential growth, yes. 


25 THE COURT: And did you do the math, or is this all 
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1 done by a -- 


2 THE WITNESS: A computer does it. 


3 THE COURT: A computer does it, okay, and so it ends 


4 up at one ina trillion. And say exactly what one in a 


5 trillion is. 


6 THE WITNESS: Can you repeat the question? 
7 THE COURT: How do you explain to a jury what one in a 


8 trillion means? 


9 THE WITNESS: You would say, what's the likelihood of 


10:25 10 seeing this if they're uncorrelated? 


11 Q. So if go back to your red "balls out of a jar" example, 


12 how does 1 work? So there's 38 balls in the jar. 


13 44 percen f them are red, and the rest are white, right? 
14 A. Yeah, but the actual math that the Fisher Exact Test does 


15 is, it finds every single possible combination of that, you 


16 know, 16,000, 38,000, and it finds every single possibility, 


17 possible combination; and it says, "How many times do you even 


18 see 343?" And it's a really big number. 


19 So Fisher Exact Test used to be that you weren't 


10:26 20 supposed to use them for these big, large samples because it's 


21 computationally expensive; it takes a long time for a computer 


22 to run. But now computers are fast, so it can do this quickly. 


23 QO. So it's measuring —- it's called an "exact test" because 
24 it's giving you the exact number of times you would see this 


25 result? 
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1 A. Right, and that's what -- the exact test is why you can 
2 set it up as a probability question. 


3 Q. Okay. So going back to the balls in the jar, there's lots 


4 of different ways of picking out 356 balls, right? You could 


5 pick out infinite combinations of 356 balls? 


6 A. Trillions, it's not infinite, but, yes, trillions. 


7 Q. But one ina trillion times, when you pick out 356 balls, 


8 343 of them will be red? 


9 A. Less than one i trillion. 


10:27 10 Q. Less than one i trillion, okay. 


11 THE COURT: Has this test been used in court before, 


12 do you know? 


13 THE WITNESS: I don't know. 


14 Q. Well, did you use it in the hacking cases that you worked 
15 on before? 

16 : Yes: 

17 : And you mentioned the newswire hacking case. 


18 : Yes. 


19 F Was it used in that case? 


10:27 20 : It was certainly submitted to the other side. 


21 remember if it was testified to. 


22 Q. Okay. But in that case, there was expert testimony in 


23 court that you supported about your statistical analysis? 


24 A. Sorry. Repeat the question. 


25 Q. In that case, you supported the expert analysis that was 
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1 presented in court? 


2 A. Yes, I did. 


3 Q. Okay. And that case did go to trial in a criminal case in 


4 Eastern District of New York? 


5 ‘é Submitted the Fisher Exact Test to the other side. 


6 . And across these traders, were the p-values consistent? 
7 : They were similarly, yeah, very small. 

8 : Okay. And Judge Saris just asked you about using 

9 software. What is the software that you use for this? 


10:28 10 2 I use SAS and I use STATA. 


11 ; And are those standard off-the-shelf software products? 


12 : Used by statisticians, yes. 


13 : Are there other tests you could have run to conduct this 


14 analysis of whether there was a relationship between the 


LS trading around earnings and who the filing agent was? 
16 A. 


17 O. Without getting into detail, what are some of those tests? 


18 A. A t-test, an ANOVA test, a chi square test, logistical 


19 regression. 


10:28 20 Q. Okay. So you mentioned a t-test. That's like the letter 


21 T test? 

22 A. T, yes. 

23 Q. A chi square test, that's c-h-i? 
24 A. That's right. 


25 . The ANOVA test, A-N-O-V-A? 


10:29 10 
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And a logistical regression? 


YES. 


You're aware that the defense has critiqued your use of 


the Fisher's Exact Test in this case? 


A. 


Q. Since learning 


of that critique, have you run these other 


tests as well? 


Yes. 


And that also used software? 


That's right. 


What was the results? 


lar result 


And just to be 


logistic regression 


££ 


10:29 20 


21 


22 


23 


24 


25 


lar results, same conclusions. 


across all those tests? 


£ 


fair, these are different tests. Soa 


controlled for market and industry 


actors. That's a different question, but I still found the 


correlation was, you know, one in a trillion. 


Os And so controlling for market and sector, that's what 


Judge Saris was aski 


ing about earlier, whether it was a 


large-cap company in the S&P 500 or a small-cap company? 


A. Right. It uses the literal market cap. 


THE COURT: 


It uses the what? 


THE WITNESS: The market cap, the shares outstanding 


times price. 
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1 ‘ So the actual market capitalization of the company? 


2 : That's right. 


3 ‘ And you also looked at what sector the companies were in? 


4 : Yes. Again, the reason why this makes sense is because if 
5 you look at the market share by market cap, it stays around 


6 44 percent. 


7 Q. So it didn't change your conclusion as to the correlation 


8 between the identity of the filing agent and the fact that the 


9 traders traded in that earnings event? 
10:30 10 A. That's right, it did not change my conclusion. 


11 Q. And the degree of statistical significance, as measured by 


12 the p-value, was similar across all these tests? 
13 A. That's right. 


14 THE COURT: Have you disclosed to the defense these 


15 other tests? 


16 THE WITNESS: Yes. 


17 Q. Now, the second way in which you used the Fisher Exact 


18 Test was to examine the correlation between trading direction 


19 and subsequent earnings surprises; is that right? 
10:31 20 A. That's right. 
21 Q. So let's break down what that exactly means. 


22 trading direction? 


23 A. Whether the trader expected the stock to go up and bought, 


24 or they expected the price to go down and shorted. 


25 Q. So your assumption was, if somebody expects the stock 
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1 price to go up, they'll buy the stock? 


2 A. Yes. 


3 Q. And if somebody expects the stock price to go down, 
4 they'll engage in what's known as a short sale? 
5 A. That's right. 
6 Q. And that's where you borrow the stock and sell it and then 
7 buy it back at a later point in time? 
8 A. 
9 Q. So that's on the one hand. And on the other hand, you 
10:32 10 said the variable was earnings surprises? 
11 A. That's right. 
12 Q. What is an earnings surprise? 


13 A. The market has an expectation to what the earning is going 


14 to be. When the earnings list comes out, the actual earnings 


15 is released to the public, and people compare the actual 


16 earnings release to what was expected. If they beat it by a 


17 lot, that's called an "earnings beat," and if they miss by a 


18 lot, it's called an "earnings miss." 


19 Q. And what was your dataset for what the market's 


10:32 20 expectations for each of these earnings announcements was? 


21 : I used IBES. 


22 : BES? 


23 


24 . -B-E-S? 


25 . Correct. 
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1 ‘ And what does that stand for again? 


2 ‘ Institutional Brokers’ Estimate System. 


3 Q. So let's just talk about earnings estimates for a minute. 


4 Who estimates how a company is going to perform quarter to 


5 guarter? 


6 A. Stocks are followed by analysts. These analysts work at 


i big banks, and they follow the particulars of the company. 


8 They do a lot of research on the company. They know all the 


9 context, and they make predictions about what the earnings is 


10:33 10 going to be. 

11 Q. And particularly they do that on a per-share basis, 
12 earnings per share? 

13 A. 


14 QO. Then what does IBES do with those individual estimates? 


15 A. It gathers them, and it gives a consensus analyst 
16 expectation. 
ay Q. And then you used the actual trading data that we looked 


18 at earlier to determine how the traders traded, long versus 


19 short? 
10:33 20 A. 
21 Q. Could we look at Exhibit G, please. Okay, so what does 


22 Exhibit G show? 


23 A. I saw a pattern in the data, and I wanted to show it. The 


24 pattern in the data is that they traded long 86 percent of the 


25 time when the company beat earnings by a lot, and they traded 
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1 long only 50 percent when the company missed by a lot. 


2 Q. When you say "they," whose trading is reflected in 


3 Exhibit G? 


4 ‘ Klyushin. 
5 : Okay, so this is just Mr. Klyushin's trading? 


6 : That's right. 


. : Did you see a Similar pattern for the other traders? 
8 ‘ Yes. 


9 : So you talked about beating by a lot or missing by a lot. 


10:34 10 do you mean by "a lot"? 
11 : So there's a third category, so it's earnings beats, 


12 earnings meets, or earnings misses. When it's an earnings 


13 meet, you don't really know what direction the stock is going 


14 to move; but if it's an earnings beat or an earnings miss, you 


15 have a better expectation. 


16 Q. Okay. So what did you do with respect to earnings meets? 


17 A. I tried to take those out of this analysis. 


18 Q. So you were just looking at the biggest earnings beats and 


19 the biggest earnings misses? 

10:35 20 A. That's right. 
21 Q. And that was on the assumption that those will be 
22 reflected most clearly in the movement of the stock? 
23 A. That's right. 


24 Q. Okay. And so how did you define that? 


25 A. I took the data and I put it into quartiles 
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i negative and positive, and I dropped the bottom quartile. 


2 Q. Okay, so you just looked at the top three quartiles of 


3 beats and the bottom three quartiles of misses? 


4 P Yes. 


5 : Okay, and you found that -- tell us again what you found. 


6 : I found that they traded long when there were big earnings 


7 surprises, and they traded short when there were big earnings 


8 misses. 


9 oO. And so, again, these are trades that they're placing 


10:35 10 before the earnings are announced? 


11 A. That's right. 


12 Q. Okay. And so you found that 86 percent of the time that 
13 they traded long, there was a large earnings surprise, a 
14 positive earnings surprise? 


15 A. Yes, a top three guartile or earnings surprise, yeah. 


16 Q. And 85 percent of the time when they traded short, there 
17 was a big earnings miss? 


18 A. That's right. 


19 Ox And sometimes, 14 or 15 percent of the time, they traded 


10:36 20 direction you wouldn't expect based on the earnings? 


21 , Based on the earnings alone. 


22 : And what did you do once you had observed this difference? 


23 : Sure. So I saw this pattern in the data. I wanted to 


24 give some context, so I ran a statistic to see if this was 


25 something that could have occurred by chance. 


Case 1:21-cr-10104-PBS Document165 Filed 01/27/23 Page 60 of 146 


60 


al : And, again, what statistic was it that you ran here? 


2 ‘ I used the Fisher Exact Test. 


i : Could we look at Exhibit 8. What was your null hypothesis 


4 time when you ran the Fisher Exact Test? 


5 : That the direction that they traded was unrelated to the 


6 direction of the earnings surprise. 


7 : That there was no correlation between those two things? 
8 : There's no correlation, that's right. 


9 : And so, again, what are Columns 1 and 2 showing? 


10:37 10 ‘ The same thing I charted. So it shows that when there was 


ala a negative earnings surprise, they rarely traded long, and when 


12 there was a positive earnings surprise, they almost always 


13 traded long, 76 to 93 percent traded long. 
14 Q. And what does the third column show? 
15 A. That's the result of my test of how often would you see 


16 this pattern in the data if you assume that, under the null 


LS hypothesis, that you assume that they're unrelated? 


18 O.. Okay. And what was your conclusion? 


19 A. That the p-value was less than one in a trillion, and that 


10:38 20 I reject the null hypothesis that they were trading by chance 


21 alone. 


22 Q. So when you reject the null hypothesis, you're saying: 


23 these two events were uncorrelated, if there was no 


24 relationship between how they traded and whether or not there 


25 was an earnings surprise up or down, you would expect to see 
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1 what you actually observed in the data less than one ina 


2 trillion times? 


3 A. That's right. 
4 Q. And just to be clear, why would the null hypothesis be 


5 that you would not expect there to be a relationship between 


6 the direction that they traded and the surprise? 


i A. Because these are surprises, they're unexpected by the 


8 market. They're movements that are unexpected by the market 


9 that are contained in the earnings release. 


10:38 10 on And that's why the stock moves afterwards? 

11 A. That's right, because it's incorporating new information 
12 into the stock price. 

13 THE COURT: Did you look at the particular companies 
14 that were being traded? 


15 THE WITNESS: I looked at some of them. 


16 THE COURT: So was that -—- the trading, could that 


17 have been predicted by what was in the, I don't know, public 


18 market or having a good analyst or consultant? Take Tesla. 


19 THE WITNESS: So, yeah, people can and do try to 
10:39 20 predict it, and it's possible to try to predict, yes. 


21 THE COURT: With financial advisors, right? 


22 THE WITNESS: Yeah, anyone who does an earnings trade 


23 they can beat the market. This just shows that in the 


24 that they are definitely beating the market. 


25 THE COURT: So a good analyst might be one good 
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1 explanation? 


2 THE WITNESS: That's right. 


i) Q. Did you try to offer an explanation, or are you just 


4 showing a correlation? 


5 : I'm describing the data. 


6 : So there could be explanations? 


7 F There are likely explanations. 


8 : One explanation might be that somebody is a really good 


9 analyst and is able to predict earnings surprises 86 percent of 
10:40 10 the time? 


11 A. That's right. 


12 ; Even though the rest of the market isn't able to do it? 


13 : That's correct. 


14 : Another explanation might be that they had inside 


15 information? 


16 : That's correct. 
17 : They knew something the market didn't know? 


18 : That's correct. 


19 : Are you offering an opinion as to which of those things 
10:40 20 explains the data? 

21 : No. 

22 ‘ You're just observing the correlation? 

23 : That's right. 


24 : Okay. The third major test that you ran was examining the 


25 start of trading time versus when a download occurred on the 
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al Donnelly Financial system, correct? 


2 : That's right. 


3 : Could you explain to Judge Saris what that means. 


4 F I received a database that showed the timing that 


is downloads of, like, Exhibit 99, among others, were downloaded 


6 from the DFIN system. 


7 : Okay. Where did you get that download time data from? 


8 ‘ It was given to me. 


9 : And it came from Donnelly Financial? 


10:41 10 : [ believe so. 


11 : And you for this purpose only looked at Donnelly 


12 Financial, correct? 


13 : That's correct. 


14 . Why? 


15 : I only had data for Donnelly Financial. 


16 ‘ Okay. So what information was in those logs? 


£ 


Ly P There was a lot o 


information, but the relevant 


18 information was that there were downloads occurring in 


19 particular companies at particular times. 
10:41 20 Q. Okay. And a download is when a document is being 
21 downloaded from the system? 
22 A. That's right. 
23 THE COURT: Let me ask you. That's curious. Why 
24 didn't the other company provide the data? 


25 THE WITNESS: I don't know. 
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i THE COURT: They didn't give a reason? 


2 MR. FRANK: IT can answer that, Judge. He's not able 
i) to answer that question. 
4 THE COURT: Well, he may not know, right. 


5 MR. FRANK: The reason is that the logs that they 


6 maintain are different at the two companies, so they just have 


7 a different degree of specificity in their log data. So why 


8 don't you just answer her questions. 


9 MR. KOSTO: By way of example, your Honor, for the 


10:42 10 dataset of access to Toppan Merrill's network, the logging that 


‘lal the company had in place when it discovered that there was an 


12 intrusion was approximately 90 days' worth of history. So most 


13 of the logs that exist for Toppan Merrill are in that 90-day 


14 window. 


15 As to Donnelly Financial, they had more robust logging 


16 going on on their network before they discovered the intrusion, 
17 so they had approximately two years plus or minus of data, 


18 which allows a more robust example for Mr. Clarke to look at. 


19 THE COURT: So the other financial analyst eliminates 


10:42 20 i fter a certain -- 


21 MR. KOSTO: I think a fair way to say it would be, it 


22 tracks for 90 days, and then it falls off the ledger so that 


23 they're not storing -—- 
24 THE COURT: So they don't have that data? 


25 MR. KOSTO: They do not have that data, your Honor. 
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i Q. So what are the variables that you were looking at 


2 purposes of this third test? 


3 A. I was looking at the timing of the download and the timing 
4 of the trade. 


5 Q. To see if there was a relationship between when the 


6 traders traded and when somebody downloaded something from 


i Donnelly Financial? 


8 A. That's right, because the expectation is that these 
9 shouldn't be related in any way. 
10:43 10 Ou And why would the expectation be that you should not see a 


1] relationship? 


12 A. I think common sense: Why should someone downloading data 


13 for their job have anything to do with traders trading in 


14 Russia? 


15 : Let's take a look at Exhibit I. What is Exhibit 


16 : This is one example of one trading episode. 
17 : Okay, so just walk us through what we're looking at here. 


18 : Sure. So this is one trade that Klyushin made in Sprouts 


19 Farmers Market, which is a grocery store chain, on May 5 and 6, 
10:44 20 2020. 
21 Q. Okay. So what happened on May 5 at 3:53 a.m.? 


22 A. At 3:50 a.m., I can see from the log file that the Sprouts 


23 Farmers Market earnings press release was downloaded from 


24 someone, a Jason Lewis account and a particular IP address. 


25 Q. So somebody using the employee username Jason Lewis from 
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i IP address downloaded the earnings release at 3:53 a.m.? 


2 ‘ ERastern, yes. 


3 ‘ What happened that same day at 11:50 a.m.? 
4 ‘ Klyushin began buying Sprouts Farmers Market. 


5 : What happened at 4:02 p.m.? 


6 : After market closed, the earnings were released to the 
7 public. 


8 Q. What happened the following morning at 9:31 a.m.? 


9 A. Klyushin began selling his shares in Sprouts Farmers 


10:45 10 Market. 


11 : And did he sell all of his shares or some of his shares? 


12 : He sold all of his shares. 


13 : So what was his total profit from that one-day trade? 


14 : 512,000. 

15 : And what was his return in that one day? 
16 : 3.1 percent. 

17 : Let's take a look at Exhibit J. 


18 THE COURT: Let me just talk about timing. How close 


19 are you to finishing? 


10:45 20 MR. FRANK: Within ten minutes. 


21 THE COURT: Okay, so we'll finish at the break, we'll 


22 have a break, and then you'll have a chance to cross. 


23 MR. NEMTSEV: Thank you, your Honor. 


24 Z So what is Exhibit J? 


25 . It's a comparison of the trading time verse the download 
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1 time, and it shows that nearly all of the trades occurred at 


2 the download time. 


3 Q. So let's put up Exhibit I on the left and Exhibit J on the 


4 right. Okay, so on Exhibit I, we see that there was a download 


5 at 3:53 a.m.? 
6 A. That's right. 


7 Q. So if we look at Exhibit J, is that going to be reflected 


8 in one of those little blue dots? 

9 : That's right. 
10:46 10 . So the download time, that's on the X axis? 
11 : Yes. 


12 : So 3:53 a.m., where would that be on the X axis? 


13 : I think that's 12 hours before, so minus 12 hours. 


14 ‘ Okay, so that's to the right a little before about 


LS right about where the cursor is? 


16 A. That's right. 


17 Q. Okay. And then the trade, the first trade by Mr. Klyushin 


18 is at 11:50 a.m.? 


19 Yeah, that's about minus four hours. 


10:47 20 : Minus four hours before the earnings announcement? 


21 : That's right. 

22 ; That's on the Y axis? 

23 : Yes. 

24 : So where would that be? The earnings announcement 


25 at the top? 
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1. : It's in the top right. 
2 ‘ That top red line is the earnings announcement? 


3 F Yes. 


4 ‘ Okay, so somewhere in let me see if I can mark it. 


5 THE CLERK: Our annotation is broke. It's not 


6 working. 
7 MR. FRANK: So, Ms. Lewis, if you could just move your 
8 cursor —— 


9 THE COURT: Is Lynn here? 


10:47 10 THE CLERK: She will be at the break. 
11 THE COURT: The break, good. 


12 MR. FRANK: So if your Honor can see where the cursor 


13 is —- oh, Ms. Lewis has been able to make a mark. I think it's 


14 probably a little to the left of that Ms. Lewis. 


15 Q. Is that roughly where this Sprouts Farmers Market trade 


16 would be represented by one of those blue dots? 


17 A. That's right. 


18 THE COURT: Where is that? I'm sorry. 


19 MR. FRANK: The top right, your Honor sees a little 


10:47 20 red circle? 


21 THE COURT: Oh, there it is, yes. 


22 Q. So that's showing, just to be clear, that the trade was 


23 about four or five hours before the earnings announcement —-— 
24 A. Uh-huh. 


25 Q. —- on the Y axis, and on the X axis, about 12 hours, that 
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1 the download was about 12 hours before the earnings announcement? 


2| A. Right. 


3 Q. Okay. And so this box is shaded, half of this box is 
4 traded in gray. What's reflected in the gray shaded portion? 
5 As That means that the trades occurred after the download. 


£ 


6 Q. In other words, the trade occurred before the earnings 


7 announcement but after the download? 

8 A. That's right. 

9 Q. And what's reflected by the fact that we see concentrations 
10:48 10 of trades up near the top red line? 


11 A. That means they often traded in the day before earnings. 


12 THE COURT: Let me ask you. I'm noticing this is in 


13 the height of the pandemic just as everything was slowing down. 


14 Is there a way in which he was trading, shorting on news about 


15 the pandemic? 


16 THE WITNESS: don't know. 


17 THE COURT: Right. You're just simply saying it didn't 


18 happen by chance? 


19 THE WITNESS: That's right, that I see a pattern in 


10:49 20 the data, and the question, is this a relevant pattern? The 


21 answer is, yes, it's a relevant pattern. 


22 THE COURT: What is Sprouts Farmers Market? 


23 THE WITNESS: One of the things he trades. t's 


24 think like an organic farmers' market, kind of a grocery store, 


25 like a Whole Foods competitor. 
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1 THE COURT: All right. 


2 MR. FRANK: Okay, if we could look at -—- well, if you 


3 could take down Exhibit I, and let's just look at Exhibit J, 
4 Ms. Lewis. 


5 Ox What was your null hypothesis here again? 


6 A. My null hypothesis was that the trade time and the 


7 download time should be unrelated. You should see no pattern 


8 in the data. 


9 Q. And that's because your assumption is, somebody deciding 


10:49 10 to trade in Moscow would have no relationship to when an 


‘lal earnings release is downloaded? 


12 ; I think that's a good expectation, yes. 


13 ‘ And what was the test you ran? 


14 : I ran a permutation test. 


15 ; What is that? 


16 : It's a test of continuous variables. 


17 : So continuous variables again being time? 


18 : That's right. 


19 : And what did the test show? 
10:50 20 : The test showed that there is a relationship between the 


21 trade time and the download time, and that relationship is that 


22 the trades occur after download. 


23 Q. And were you able to calculate a p-value? 
24 A. 


25 QO. What was that? 
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dL: A. For all traders, I think it was less than one ina 
2 million. 
3 0. One in a million? 


4 A. That's right. And this test requires me to runa 


5 Simulation a million times. 


6 Q. Okay, so you ran it a million times, and you found tha 


7 you do this a million times, you'll find this pattern less 1 
8 one in a million times? 


9 A. That's right. 


10:51 10 Q. And essentially what it shows is, even when the trade 


11 occurred before the earnings announcement, the download 


12 occurred even earlier? 


13 A. That's right. Even when they traded three days 


14 beforehand, the download occurred three days beforehand. 
15 ; Or more? 


16 - Or more. 


17 : So in almost no instances did they trade after a download? 


18 : That's right. 


19 ; The download almost always occurred before? 

10:51 20 : That's right. 
21 : And that's why it's in the gray box on the left? 
22 : That's right. 


23 : Okay. Now, this is for all traders? 


24 ‘ That's right. 


25 MR. FRANK: Can we bring up Exhibit K on the right, 


10:52 10 
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A. 


Q. 


Okay, 
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what's in Exhibit K on the right? 


The same analysis for Klyushin alone. 


And what did you observe about how Mr. Klyushin's trading 


versus download time compared with the other traders? 


Similar. 


And what was the p-value for Mr. Klyushin? 


It was 17 in a million. 


Seventeen out of a million? 


That's right. 


It is. 


Is that also statistically significant? 


What would this chart look like if there was no 


relationship between trading and download time? Where would 


10:52 20 


21 


22 


23 


24 


25 


those dots be distributed? 


A. 


Your expectation, if they were unrelated, is that they 


should be distributed kind of throughout the box. You should 


seea 


Q. 


download, 


And 


little cloud 


in every little portion. 


that's because sometimes he would be trading before a 


sometimes he would be trading after a download, and 


it would be distributed all over this box? 


A. 


thing as it 


YESSie 


You would split up the number of dots across the 


relates to the distribution of the download time. 


MR. FRANK: You can take that down. 


One of the det 


fense criticisms of your analysis was that 


10:53 10 


10:54 20 


21 


22 


23 


24 


25 
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you did not analyze trading before January 1, 2018. 


A. 


Q. How many traders are you aware of of the eight that had 


accounts prior to January 1, 2018? 


A. I'm aware of one trader. 
Q. And who was that? 


A. Sladkov. 


Q. So did the defendant, to your knowledge, have a brokerage 
account for which you had records prior to January 1, 2018? 
A. No. 


Ox Another of the defense criticisms of your analysis was 


that you stopped analyzing the trading data after September 30 
of 2018? 


A. 


Q. Could we look at Exhibit I I'm sorry, 


My mistake. What is Exhibit L? 


A. This is a graph of the dollars traded by month. 


Q. Okay. And so in the blue, it's earnings events handled by 


Toppan Merrill? 


I'm sorry. The green is Toppan Merrill. 
The green is Toppan Merrill? 
Yes. 


And the blue is Donnelly Financial? 


That's right. 


And the gray is other? 
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2 0: And here are we looking at all trading or just trading 
3 around earnings releases? 


4 ‘ This is just earnings releases. 


5 : And this is for all eight traders? 


6 : Yes. 


7 ‘ And what does this chart show about the frequency of 


8 trading? 


9 A. It shows that there is little trading at the beginning of 


10:54 10 2018, and that grows over time, and then it declines. 


‘lal ; And what happens in September of 2020? 


12 : There's no trading in September of 2020. 


13 : There's no earnings-related trading that you saw? 


14 : That's right. 


15 : It just drops off a clif 


16 . Uh-huh. 


17 MR. FRANK: Ms. Lewis is reminding me 


18 Exhibits J through L. 


19 (Exhibits J and L received in evidence.) 


10:55 20 Q. Since the defense has critiqued you for not looking at 


21 September 30, have you taken a look at those results? 


22 : I have. I did not do my full analysis because it's 


23 intensive, but I have looked at it. 


24 Q. And without doing a full analysis, what was your 


25 observation? 
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1 A. There were only eleven traders -- I'm sorry -—- only eleven 


2 accounts, and the trading is less and the profits are less. 


3 O.. So they trade less and they make less money? 


4 A. And they make less money per trade. 
5 MR. FRANK: Thank you. 


6 THE COURT: Was that disclosed to the defense? 


7 MR. FRANK: There's no formal analysis, Judge. He's 


8 just saying what he did. 


9 THE COURT: Back-of-the-envelope kind of thing? 


10:55 10 MR. FRANK: He's just looking at and seeing less 


‘lal trading and less profitable trading. He didn't do an analysis. 


12 THE COURT: I'm not sure he can testify to that 


13 without disclosing it. 


14 MR. FRANK: I have no further questions, your Honor. 


15 THE COURT: All right. So I'm just saying that if you 


16 want to testify about it, you just have to disclose the math. 
17 MR. FRANK: Understood, your Honor. 


18 THE COURT: Okay. 


19 Now, would you like to take a break? 


10:56 20 MR. NEMTSEV: Yes, please, your Honor. 


21 THE COURT: And so what are you anticipating? Are you 


22 going to full cross right now? 


23 : EMTSEV: Yes, I think I will. 


24 THE COURT: Okay. So why don't we come back here 


25 around 11:20-ish. Does anybody have any conflicts? 


10:56 10 


11:31 20 


21 


22 


23 


24 


25 
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MR. FRANK 
THE 


do you think 


COURT: 


you might need? 


No, your Honor. 


So come back here around 11:20. 


know it's hard to predict. 


MR. NEMTS 


EV: 


THE 


COURT: 


Forty-five minutes. 


Oh, okay. And what do we do about your 


We don't know yet? 


MR. NE 


IS 


EV: 


THE COURT: 


We don't know yet. 


First of all, 


he's sake, and second of 


don't know if you need him? 


MR. NEMTS 


EV: 


THE COURT: 


That sounds great. 


MR. FRANK 


Yes. 


Okay, so we'll come back around 11:20. 


Thank you. 


Thank you. 


MR. NE 


TS 


EV: 


THE CLERK: 


(A recess was taken, 


(Resumed, 


THE COURT 


Thank you, your Honor. 
All rise. 


10:57 a.m.) 


Lirvs0 a.m.) 


Okay, go ahead, sir. 


MR. NEMTS 


EV: 


CROSS-EXAM 


NAT 


Thank you, your Honor. 


ON BY MR. NEMTSEV: 


Q. 


represent 


THE COURT 


Good afternoon Mr. 


MR. NEMTS 


EV: 


Clarke. 


My name is Mark Nemtsev. 


You're way too soft. 


Oh, God, judge I try and I 


T'll get it right by the end of this trial. 


76 


How long 
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1 j Mr. Clarke my name is maximum Nemtsev. I represent the 


2 fendant, Vladislav Klyushin, in this case. 


3 : Nice to meet you. 


4 : ice to meet you? 


3 ‘ Great name. 


6 : Thank you. Likewise. Up on the board is your resume’; is 
- correct ? 
8 : Correct. 


9 ‘ That is something that you submitted in connection with 


11:31 10 your analysis and your expert disclosure which you signed, 


de. correct? 


12 A. Yeah. I have an updated resume as well. 


13 Q. What are the differences between the /SR-RS and this 


14 resume ? 


15 A. [ think I changed my title to senior financial analyst to 


16 reflect the name change, and I think the -- well, I don't know 
LS] every difference. 


18 Q. Understood. Going through your educational background, 


19 you have a bachelor of arts cum laude in economics mathematics 
11:32 20 minor, correct? 
21 A. That's correct. 


22 Q. Graduated in 2011, approximately 12 years ago at this 


23 point? 
24 A. That's right. 


25 Q. You don't have a major in statistics but you focused your 
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i course work according to your testimony in statistics, right? 


2 A. That's right. 


3 Q. Now, your professional experience and employment, you're a 
4 research associate at Cornerstone? 


5 A. Yes senior analyst and associate yeah (). 


6 Q. After that you exclusively worked for the Securities and 


7 Exchange Commission, correct? 


8 A. That's correct. 


9 Q. First as a financial economist and then a senior financial 


11:32 10 economist? Sorry? 


11 : First as an economist then a financial economist. 


12 : And then subsequently a senior financial economist? 


13 : That's right. 


14 ‘ And you work in this division of economic and risk 


15 analysis, the Office of Litigation Economics, correct? 


16 A. That's correct. 


17 Q. And essentially what you do, what your task is to support 


18 prosecutors to support prosecutors from the SEC in litigating 


19 these cases, correct? 


11:33 20 A. I do the data work for them. I like to say I do math 


21 lawyers sometimes. 


22 Q. And have you ever assisted a defendant or a defense 


23 attorney in connection with their analyses? 


24 A. When I worked at Cornerstone Research. 


25 Q. And what did you do at Cornerstone Research? 
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1 A. Will I was -—- essentially the same stuff. I summarized 


2 data draw conclusions from that data using statistics, among 


3 other things. 


4 ‘ Did you use the Fisher Exact Test? 


5 F Tt -did.. 


6 ‘ The non-permutation —- 


i : I never used a permutation test at Cornerstone Research. 


8 : Understood. Did you testify behalf the defense while you 


9 at /SR-RS research? 


11:33 10 : No, but my job was to support experts. 


11 : Understood. Now, you said you testified in the Sl 
12 Sergeant case? 


13 : That's correct. 


14 : Before Judge Young in this courthouse? 


iS) : [ did it via Zoom, but, yes. 


16 : And what was the issue that you were testifying on? 


17 : The issue had to do with the feasibility as to whether or 


18 not you could calculate harm to shareholders, in light of the 


19 Supreme Court decision of Liu. 


11:34 20 Q. Which deals with disgorgement issues? 
21 A. That's right. 


22 Q. It wasn't the comparison of whether or not a trading 


23 correlates to, for example, a filing agent or something else, 
24 correct? 


25 A. That's correct. 


11:35 10 


11:35 20 
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You featured some selective case work, correct? 


Sure. 


O.. You worked an insider trading case where you calculated 


ill got even gains; is that correct? 


(Witness examining document.) 


Uhm, yeah, this —- 


got even gains from 


this seems off to me for some reason. 


submitted a new CV that would be more correct. Calculating 


trades made with material nonpublic 


And is this the SEC 


No. 


No. 


formation," that's right, that's right. 


v. Sergeant case? 


Did you testify in connection with this case? 


Did you submit an expert report in connection with this 


but I supported 


Was there a Daubert 


the expert on this case. 


hearing on this case? 


don't think so. No. 


Q. Did anybody ever testify to the analytics that you've done 


in this case? 


21 


22 


23 


24 


25 


A. In the -- this was the news /SR-RS hack yes someone did. 


Q. Going to the newswire hack, this is the calf chess key 


case in the district of New York? 


A. Yes, EDN Y. 


Q. So you weren't the main status in that case in that case? 
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al A. I don't know what you mean /SR-RS. 


2 Q. You weren't the head witness presenting any expert 


3 testimony on the subject matter, correct? 


4 . Correct. 


5 : You just supported the statistician in that case? 
6 : Yes. 


7 : You helped submit an expert report in that case? 


8 : I don't think there was an expert report submitted in that 


11:36 10 : So no one testified as an expert in that case, correct? 


11 : No. Someone testified as an expert in the newswire 


12 hacking case. 


13 : But did they never submit an expert report? 


14 ; It's criminal so you don't submit an expert report. 


15 : No one submitted an expert disclosure? 


16 THE COURT: By the way, that's not necessarily so. 


17 THE WITNESS: Yeah, sorry, I'm not a legal -—- I don't 


18 know the answer. 


19 Q. Was there a Daubert hearing similar to this one to 


11:36 20 determine whether the methodologies used in that case were 


21 accurate, to your knowledge? 


22 A. To my knowledge, no. 


23 Q. And were the statistics that you present here, the one in 
24 a trillion chance, one in a million case, were those admitted, 


25 to your knowledge, in that case? 
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1. : What do you mean by admitted? 


2 ‘ Meaning was that evidence presented to a jury, deciding 


= 


a criminal defendant? 


3 fate o 


4 A. Were the words were the words "one ina trillion ooh 


5 used? Is that your question? 


6 Q. Was that evidence present to the jury? 


7 A. I don't -- I don't know. I don't think so. 


8 Q. Thank you. Do you know any criminal case that is serviced 


9 by your litigation assistance department where similar 


11:37 10 statistics, the Fisher Exact Test and the other tests that you 


11 used, were admitted to prove the correlation of events? 


12 A. In the newswire hack case, the permutation test was used 


13 about timing; and in the EDGAR hack, we used a Fisher Exact 


14 Test in avery Similar -- 


15 : But in the newswire case you just testified that you don't 


16 whether those statistics ever made it to the jury correct? 


17 : I'm sorry I was talking about the Fisher Exact Test the 


18 permutation certainly did. 


19 OQ. And /SR-RS? 


11:38 20 A. The timing of the download versus the */* the timing of 


21 the up load to the newswire service and the timing of the 


22 trade. 


23 Q. Understood but you don't know whether the Fisher Exact 


24 Test or you believe that the Fisher Exact Test wasn't admitted 


25 in that case is this /(? 
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al A. Uhm, I don't -—- I don't have a full recollection of this, 


2 but I know that we submitted an exhibit that had a Fisher Exact 


3 Test in it. I don't know if it was actually -- I don't know 


4 the distinction in your mind of testified verse not testified 


5 and submitted. I don't know these distinctions. 


6 Q. Understood, understood. So, now, moving on-and when were 


7 first assigned to deal with this case? 


8 : This case? 


11:39 10 : late August because it was at I came back from 


11 paternity leave. 


12 Q. In late August of 2022? 


13 A. Of 2022, yes. 

14 Q. Okay, was there anybody working on this case from your 
LS division before that? 

16 A. 


Ly QO. And who was that? 


18 A. ... L think the names were Katie sheen foster and 


19 consistent or are Hurley? Yeah, yes, consistent or Hurley. 


11:39 20 : Who gave you the dataset to analyze in this case? 


21 : I did the -- I got the -- I did it myself. 


22 : You acquired the dataset yourself, meaning -—- 


23 : Specifically I checked. There were 30 Excel files. 


24 uploaded them into SAS which is mill statistical program. I 


25 compiled the data by myself, and then I did all -- I did it all 
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1 myself. 
2 Q. Who asked you to limit your analyses from January, 2018, 


3 to September of 2020? 


4 A. Who asked me to do that? The lawyers on the case. 
5 O. So Mr. Frank and Mr. Cost asked you to limit your analysis 


6 specifically to the conspiracy period correct? 


7 A. Yeah. I asked what is the time period for this analysis 


8 would happen. They said 11 through 930. 
9 Q. So you excluded any trading that was done by these 


11:40 10 defendants or prior to January, 2018, any trading that they did 


1 after September of 2020, correct? 


12 A. I excluded yes trades outside of that time period from my 


13 analysis. 


14 Q. And would you agree that having -- and I think you said 


LS this -—- large sample sizes, more data is good in statistics? 


16 A. It depends on the circumstance, but typically, yes. 


17 Q. So was there any reason to arbitrarily -- strike that. 


18 there any reason to limit this data to exclude trading before 


19 January of 2018 and after September, 2020? 


11:41 20 A. I think it has to do with the other evidence that might be 


21 presented in this case. I don't know. 


22 Q. But it would be relevant tower analysis, for example, if 


23 there's no hacked, meaning no one accessed any reports of 
24 device inn and yet of there was trading concerning device 


25 related stocks and trading /SR-RS? 
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1 MR. FRANK: Objection assumes facts not in evidence. 


2 MR. NEMTSEV: Your Honor he's an expert he's allowed 
Si to assume facts. 


4 MR. FRANK: He's not an expert to when there was a 


6 THE COURT: Well, what is the hack you want him to 


7 assume. 


8 MR. FRANK: I think there's an assumption of when 


9 there was a hack. 


11:41 10 MR. NEMTSEV: The prosecutors made that assumption for 
11 him. They gave him the time period when they allege was a 
12 hack. 


13 THE COURT: What's the question again? 


14 MR. NEMTSEV: The question is wouldn't it have been 


15 relevant to your statistical analysis to have trades in filing 


16 agent or device inn related client stocks /( where there was no 


17 hack or evidence of a hack, yet there was trading by these 


18 traders around earnings reports of those companies? 


19 MR. FRANK: My objection, your Honor is. 


11:42 20 THE COURT: /SR-RS go ahead. 


21 A. I wasn't really aware of when the hack occurred. I was 


22 given the time period. It wouldn't affect my analysis. Answer 


23 a different question. The question I'm answering is, does this 


24 relationship exist during the time period that I looked? 


25 you had a different time period, it would answer a different 
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1 guestion: Does the relationship exist over that different time 


2 period (). 


3 O.. Assume that a relationship exists over a different time 


4 period, wouldn't that be relevant to show that it's not whether 


5 there was a hack and information was obtained through a hack 


6 from the filing agents. There's another factor influencing the 


7 trading decisions or influencing what stocks to trade. 


8 A. Sorry, can you repeat the question. 


9 THE COURT: In other words, if you saw the same 


11:43 10 conduct before that time period, would that have a material 


11 impact on your opinion? 


12 THE WITNESS: No. It would raise a question as to was 


13 there a hack during that period or not? f found the same 


14 result including the preperiod, I would find -—- I didn't do the 


is) analysis. I don't know the answer, but if I were to find it 


16 there would be something about the trading strategy during t 


£ 


17 time period that was favoring device inn and Toppan. It 


18 doesn't change my analysis. 


19 Q. So to put together your data you used the /S*EGS numbers 
11:44 20 correct? 

21 : That's correct. 

22 ‘ You did not use a client list? 


23 : I did not /SR-RS. 


24 : And that could lead to a lower number of stocks associated 


25 earnings reports associated with filing agents 1 and filing 
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i agents 2 than those filing agents actually handled, correct? 


2 A. I didn't do that analysis. I don't know the answer. 


3 Q. Well, you did review in the first quarter of 2019 stocks 


4 or subpoenas that were clients of Filing Agent 1 and 2 who used 


5 these filing agents to compile this report, yet they filed it 


6 by other means, sometimes by themselves but they just didn't 


7 request the filing agents to file it correct? 


8 : I did do that yes. 


9 : And you only did that for one quarter? 


11:44 10 : I did only do that for one quarter. 


11 OF And how many stocks did you find that weren't filed by the 


12 filing agents but were actually served by the filing agents? 


13 A. Do you want me to go into detail. 


14 THE COURT: I was going to ask the same question. How 


15 do you understand the term serviced? 


16 THE WITNESS: So I'm happy to go through my process () 


17 (question above) so to answer your question like what's in the 


18 other -—- I saw 44 percent was DFIN and Toppan as when you use 


19 the /S*EGS number. I looked attain was wondering well, what's 


11:45 20 in the other 56 percent, why is it so big? A lot of those were 


21 ] . © also knew that there was a different company that's 


22 1] Workiva that if you look at the industry, it has a large 


23 share. I say why am I not seeing this in the data? So 


24 I went to the Q 12019 to try to see what that was. It involved 


25 a lot of looking through filings. So I looked through the 
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1 filing. I went through the filing and I saw there's a little 


2 part of the filing that will give you a, like it will say "This 


looked and I counted 


3 document was created by Workiva." So 


4 all the time it said Workiva I counted all the times it said 


5 something else. I found that most of them, I don't remember 


6 the exact numbers, but I think about ninety percent were 


7 Workiva which aligned with my expectation from what the market 


8 share said, and I think about 2 percent were from Toppan 


9 Merrill and there were zero percent from device inn. 


11:46 10 Q. And that's only your analysis as to the first quarter of 
11 2019? 
12 A. That's right. 


13 Q. You didn't do the same analysis for the remaining ten or 


14 eleven quarters that you're opining on? 


15 : I have no reason to believe that the market share of 


16 filing changed over time. 


17 Q. But you would agree that at minimum, those 44 should have 


18 been counted in your figures, correct? 


19 THE COURT: I am strange to hear you. I'm sorry. 
11:47 20 Q. Would you agree that it would be more accurate to include 


21 those 44 in the number of filing agent and Filing Agent 2 or 


22 device inn and Toppan companies that service stocks? 


23 A. I use /S*EGS number. I wanted to have an apples to apples 


24 comparison, right? So there existed times in that 96 times I 


25 think you guys pointed out avenue net, that would have /( -- 


11:48 10 
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have to include that on the other side too. 


consistent, 


number and my concl 


that were self 
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Filed but you could look at the document and see 


Toppan Merrill. You would have to include that in 


which I thought was most important, 


lusion then has to be -—-- and I think 


the 96 percent so that 96 could have been 98 percent but you 
So to keep it 
I used /S*EGS 


Tm 


pretty clear about that -- my conclusion is that is the trading 


of them associated with the DFIN and Toppan earnings as 


ascribed to 


by /S*EGS number. 


£ 


Ox And you excluded them from both ends meaning you excluded 


£ 


avenue net from the thumb of companies /SR-RS you excluded it 


from Mr. Klyushin's trading? 


11:48 20 


21 


22 


23 


24 


25 


A. I think exclude is the wrong word. 


to include. 


QO. But wouldn't it 


of both filing agent 


A /S*EGS number does not incl 


companies serviced by these two? 


A. Sorry. 


lude those. 


Could you repeat the question. 


chose to use /S*E 


t be more accurate to use the filing lists 


ts to determine the number of stocks or 


Ox Wouldn't it be more repeat, meaning a filing agent /SR-RS 


responsible 


for everything related to an earnings report other 


than the filing of it and that filing agent would be excluded 


from your total calculation, correct? 


A. Yeah, 


question —-— 


do exclude those, /SR-RS include them. 


(Discussion off the record.) 
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ac THE COURT: Let me just ask, was avenue net a 


Z f-report? 


Si THE WITNESS: Avenue net was a self-report, yes. 


4 THE COURT: How did that show up in your analysis? 


5 THE WITNESS: Self-percent would be in the 56 percent 


6 and it would be in the 4 percent. 
7 THE COURT: Both. 


8 THE WITNESS: Both keep apples to apples. 


9 Q. But you would also agree that the vast majority 


11:49 10 Mr. Klyushin did not trade in the vast majority of stocks that 


11 were serviced by these two /TPAOEKULGTS? 


12 A. Yes. He traded in a subset of the stocks. 


13 Q. A very small percentage, under 20 percent, correct? 


14 : I don't know the answer to that, but I believe you. 


LS ; Do you know how many stocks file earnings reports with the 


16 EC in a given year? 


17 A. I think it's four to five thousand of U.S. exchange 


18 stocks. 


19 Q. And how many stocks did you identify that Mr. Klyushin 


11:50 20 traded? 


21 A. Over the time period? I think it was -- it's in my 


22 exhibit but it's 350 something. 


23 : Earnings related, but how many total? 


24 é Did he trade in? I don't remember. 


25 : You excluded from your analysis any time that he traded in 
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1 another company that wasn't serviced by the /TPAOEUGTS, 


2 correct? 


3 A. No. The reason why it's 96 percent is because the other 


4 4 percent means that it wasn't DFIN and Toppan. 


5 QO. But it's not 4 percent, correct? 

6 : What? 

7 Q. Hold on. Let me put this up. Maybe it will be a little 
8 clearer. So this is Mr. Klyushin's account, correct? 


9 A. Correct. 


11:51 10 O%. So you say that he traded in 343 DFIN and Toppan Merrill 
11 events traded by trade? 

12 A. That's correct, yes. 

13 Q. And that he traded in total events traded by traders 

14 /SR-RS? 

15 A. Right. 


16 Q. You don't include in your numbers any trading that isn't 


ay related to DFIN and Toppan, correct? 


18 A. There are 13 trades that are nonDFIN and Toppan. That's 


19 343 minus 356. 
11:51 20 Q. So 13 of those are not included, and you also don't 


21 include any trades that are outside of -—- 


22 A. What do you mean are not included? I don't -- 


23 Q. 56 trades that's all that Mr. Klyushin traded -—- 


24 A. These are earnings trades these are earnings trades, yes. 


25 Q. These are earnings trades. How many nonearnings trades 


11:52 10 


11:53 20 


21 


22 


23 


24 


25 
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A. No nonearnings trades, zero. I don't include nonearnings 


trades. 


Q. So you excluded approximately 30 percent of his trading 


from your analysis? Is that accurate? 


A. I mean, I don't exclude it. I show it in that first 


exhibit. 


And this is the first exhibit you're talking about? 


¥YeSi< 


Exhibit B? 


Yes. 


And these are all of the transactions that you didn't 


incorporate into your Fisher Exact Tes 
you've done? 


A. That's correct. 


t or the other test that 


Q. It says if they never happened according to your analysis, 


correct? 

A. Well, sorry, sorry that's wrong. 
don't include the 119 and the 735. 

Q. So out of the 2,892 transactions, 
1,000 of them during the relevant time 


analysis, from your statistics; is tha 


A. No it's because we're talking abo 


trades about earnings. These are not 


The whole point of this is to talk abo 


include the 50. 


you excluded almost 


£ 


from your 


period 


cE correct? 


ut earnings trades so 


trades about nonearnings. 


ut earnings trades so I 


11:54 10 


11:54 20 


21 


22 


23 


24 


25 
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Q. But 
to try to 


data that 


nonfighti 


A. So £ 
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an analysis on earnings for nonearnings trades. 


isn't it more accurate to use as much data as you can 


get the most accurate static you can by incorporating 


"s in front of you, including his trading this 


ng stocks in /SR-RS? 


did incl 


ude the 50 () and th 


incorpora 


they didn 


te it, right, because 


"Lt trade in earnings, 


irst I don't know why you didn't circle the 50 because 


e the other on 


I'm saying that 


so I couldn't fi 


number related to that trading because it didn' 


earnings 
want me t 


To t 


related to. How coul 


o incorporate it. 


he contingency table, 


d I incorporate i 


for example, in 


es I couldn't 
they traded —- 
nd the /S*EGS 


t have an 


t how do you 


the Fisher ] 


correct you can add as many variables as you want? 


Q. So you can add, 


can add as many variables as I want is t 


for example, every instanc 


his okay. 


e where you 


conducted a trade that wasn't this a filing agent related 


stock? 


A. Coul 


d I have done that? 


could have. 


Uhm, I suppose —--— 


suppose 


Q. And you could have also added to your contingencies table 


every ins 
this a no 
A. So, 


about it. 


tance where he didn't trade in a -—- the way he traded 


nearnings report? 


yeah, this is pretty 


don't know how -- 


hypothetical. 


actually I can 


haven't thought 


"t know the 
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Say 


i answer off the top of my head. If you did the analysis, you 


2 could do it. I'll give you the data. /(. 


3 Q. What null hypothesis did you formulate in connection with 


4 your first opinion and the fact that -—- 


5 A. The null hypothesis is that the correlation between the 


6 traders trading and the identification of the earnings as being 


7 DFIN and Toppan was uncorrelated. 


8 : Uncorrelated meaning that it was essentially by chance? 


9 . That's another way to put it. 
11:55 10 : So you were looking at whether Mr. Klyushin trades 
11 randomly in companies? 
12 ; No. 


13 : No. This is Exhibit F, correct? 


14 . Yes. 


15 , So percent of earnings events that are DFIN and Toppan 
16 related events, 44 percent, right? 


17 A. That's right. 


18 Q. I'll represent to you that if you were to look at the 


19 client list, it would be approximat 64 percent, but let's 


11:56 20 stick with 44 percent. 
21 A. How did you determine that? 


22 Q. Seeing whether ever, in that time period, a company used 


23 or employed a fighting (filing agent)? 


24 A. So these are the percentage of trades or the percentage of 


25 companies? 
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i THE COURT: I must say you're having a private 


2 conversation that I'm not following. 


3 MR. NEMTSEV: "m sorry, Judge. Let me try to clear 


4 it up as much as I can. 
5 0. You used the /S*EGS number correct? 
6 A. That's correct. 


. Q. To calculate 44 percent. Had you used the market share, 


8 meaning whether the company is a client of the /TPAO*EUGT it 
9 would have been approximately 64 percent (had you used)? 


11:56 10 A. I don't know that. 


11 Q. But let's say you have 44 percent of the market is related 


12 to DFIN and Toppan, correct? 
13 A. 


14 Q. So you assume that if Mr. Klyushin was trading at random 


15 /SR-RS, for example, that his trading would have resulted in 


16 44 percent in DFIN and Toppan related companies, 56 percent of 


17 his trading would be in other entities? 


18 A. That's incorrect. Under the null hypothesis that the 


19 trading is uncorrelated the probability of seeing a pattern 


11:57 20 during of 96 percent or more extreme the probability of that 


21 would be less than one in a /TR-L. 


22 Q. So what's this figure the expected number of events both 
23 device inn and Toppan Merrill events and traded by trader 
24 /SR-RS? 


25 A. That would be your expectation under the null hypothesis. 
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1 Q. And your expectation under the null hypothesis is that for 


2 Mr. Klyushin's transactions, you would expect that he would 


3 only trade 156 times in Filing Agent 1 and Filing Agent 2 


4 stocks; is that correct? 


5 A. It's saying that the mean of your expectation of the null 


6 hypothesis is equal to 170. 


7 . Not 170. 156 for Mr. Klyushin, correct? 


8 : I'm sorry, 156, yes. 


9 : So is 15644 percent of 
11:58 10 : Yes. 


11 , Okay, so you expect that 44 percent of Mr. Klyushin's 


12 trades should have been in DFIN and Toppan service stocks, 
13 correct? 


14 A. No. Repeat the question. 


15 Q. The expected number of events both DFIN and Toppan Merrill 
16 events trader by trading fighting and trading activities are 


Ly uncorrelated correct? 


19 : This number is 44 percent of 356? 
11:58 20 A. That's right. So if you had a company, right, and you 


21 wanted to know what filing agent did they trade this, what's 


22 your best guess of what the answer to that is? I would look at 


23 the overall market share of the earnings. So I'd say if all I 


24 knew under the technological hypothetical assuming they're 


25 unrelated is that they traded this it and I'm assuming that 
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i there's no relationship, my best guess is 44 percent. So 


2 that's what I'm doing. 


3 : But that assumes that Mr. Klyushin would trade by chance, 


4 i this? 


5 “ It assumes that the relationship is, like, all statistics 


6 has a null hypothesis, and the null hypothesis is typically 
7 that there's no relationship. The null hypothesis is a random 
8 process. 
9 : So the null hypothesis assumes randomness, correct? 
11:59 10 : All statistics is based in randomness, yes. 


11 ‘ But is picking stocks a random event? 


12 : No. 


13 , Do you yourself trade in stocks? 


14 : I'm barred from doing so by the SEC. 


LS , But you have family members, friends that trade in stocks? 


16 ‘ I know people who trade in stocks. I have previously 
17 traded in stocks in my life. 


18 Q. And you didn't just randomly select the stock correct? 


19 A. That's correct and I don't say that he randomly /SR-RS my 
12:00 20 conclusion is that he isn't trading randomly. 


21 Q. Your conclusion is, the only reason that he's selecting 


22 these stocks is because they're related to fighting 1 and 2? 


23 A. That's not correct. My conclusion is that the pattern and 


24 the data shows that whatever thing he was picking, whatever 


25 strategy he was using happens to have this data signature that 


12:01 10 


12:02 20 


21 


22 


23 


24 


25 
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is 97 percent t 


of reasons. 


reason. 


rading DFIN and Toppan. It could be for a lot 


don't know the reason and I'm not opining on 


98 


Q. But you didn't attempt to discern any other reasons, 


correct? 


A. I did look 


guys submitted, 


sector. 


at what he traded in, and, you know, after you 


did look at market share and I 


looked at the 


Q. Is my calculation correct that about 75 percent of the 


market of the S&P 500 related stocks serviced by /TPAOEUGTS 1 


and 2? 


A. I don't th 


or -—- so one th 
QO. Go ahead. 


A. So in the 


If you lo 


court tile you 


ing I had to look -—- well, no. 


What did you look for? 


ink that's correct, but I haven't done the math, 


data I submitted to you guys, there's a market 


ok at quartile if you look at the market cap by 


can run and see if the 44 percent changes over 


quartile and I 
42 percent. Bu 


trillion to $15 


look at it does 


appear to restr 


t he trades this stocks that are 
OQ million. The trading strategy 
n't appear to be, like -- market 


ict the trades that he's willing 


that makes sens 


Cs 


Q. It's only market cap? 


A. That's tru 


e it's only market. 


think it's like 45 percent 46 percent 


£ 


rom like 1.7 


if you just 
cap doesn't 


to trade on, if 


12:02 10 


12:03 20 


21 


22 
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A. There is some trading strategy that he used I 
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agree and 


whatever /SR-RS strategy he used whether it be /SR-RS market 


cap I would love to see some documentation as to 


trading strategy was because then you could 


data, but 


EUGTS 


In my 


don't know what the trading strat 


and 2, correct? 


£ 


first analysis, yes. 


test 


CEGQY 


And that was at the request of prosecutors? 


No. 


what his 
it against the 


was. 


But the only thing that you correlate it to is the use of 


Whose request was that to use /TPAOEUGTS 1 and 2? 


The fighting 1 and 2, yes, they asked me the question, is 


their tradi 


ing correlated. 


Now you've got a p-value of less than point 0001, correct? 


That's right. 


So what is a p-value is this? 


A p-value is the probabil 


know -- as observed or something more extreme. 


ity of seeing a result as, you 


If you assume 


that the two -- if you assume that they're unrelated. 


Q. Would you agree that there's substantial literature 


regarding p-values and the risk of using them as a percentage 


23 


24 


25 


of chance? 


A. What? 


written on 


don't understand the question. 


p-values, yes. 


There's a lot of 


Case 1:21-cr-10104-PBS Document165 Filed 01/27/23 Page 100 of 146 


100 


i : A lo itten about p-values, but specifically there's 


2 lso a lo itten /( about the risks of relying on p-values to 


3 laim or support the results of no practical significance. 


4 ‘ What do you have up on the screen? 


5 MR. NEMTSEV: This is Exhibit 3, your Honor. 


6 COURT: Your Exhibit 3. Have you seen this 


8 ITNESS: They sent it last night at midnight, 


12:04 10 MR. NEMTSEV: was busy working. Sorry. 


11 THE COURT: Well, do you want to take some time. Have 


12 you had a chance to read it is this. 


13 THE WITNESS: I haven't read the whole thing. 


14 read this section the highlighted sections. 


LS MR. NEMTSEV: highlighted sections your Honor that 


16 he could review /THA*R relevant to the subject. 


ike) THE COURT: Hopefully you're going to finish this 


18 morning at least with this witness. 


19 MR. NEMTSEV: Understood, your Honor. 


12:04 20 Q. So this section talks about using sample sizes over 10,000 


21 observations, correct? 

22 A. 

23 Q. And obviously it says there's large advantages or many 
24 advantages to large samples? 


25 A. That's right. 
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i : But the issue is p-values guickly go to zero? 


2 . I see that. 


3 : And did your p-values quickly go to zero in this case? 


4 ‘ Quickly go -- my p-values are very small. Do they 


5 quickly -- guickly go to zero I don't think you understand what 


6 they're saying. 


7 Q. Your p-values are incredibly small. They provide a one in 
8 a trillion chance? 
9 A. That's right, yes. 


12:05 10 Q. And this article says that that's not necessarily the best 


11 way to support results, correct? 


12 A. Results of no practical significance, yes. 


13 what practical significance means? 


14 Q. Can you tell us? 


15 A. Yes. So practical significance is, is the answer 


16 meaningful? So one way that you deal with this type of problem 


17 is that you chart the data. So I would argue that 96 percent 


18 being different than 44 percent is of practical significance. 


19 Q. Have you also had a chance to look at the American 


12:06 20 statistical association release statement on statistical 


21 Significance and p-values"? 


22 A. This one I'm aware of. When it came out, it was a pretty 


23 big deal. 


24 Oz It came out in March of 


25 A. That's right. 


12:06 10 
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Oh, boy. 


THE COURT: It's blurred here. 


MR. NEMTSEV: 'm trying, your Honor. 
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THE COURT: Is this paper generally accepted in the 


Ism yes. 


THE COURT: This would be somewhat like a learned 


treatise, in particular —-- 


THE WITNESS: This is a well-known thing. Yeah when 


it came out we had discussions about it at work. 


Q. The second principle p-values don't measure the 


probability that the studied hypothesis is true or the 


probability that the data were produced by random chance alone" 


is this? 


12:07 20 


21 


22 


23 


24 


25 


A. 


Q. So it's difficult to use the p-value or the one this a 


trillion p-value to estimate percentage chance, correct? 


A. So I want to talk about this statement because 


you're interpreting it wrong. I can't see it, though 


it's kind of blurry. (Audio back). 


THE WITNESS: So the way to interpret this s 


think 


, because 


tatement 


is p-values do not measure the probability that the s 


tudied 


hypothesis is true because p-values measure the probability 


whether or not the studied hypothesis is false. And that's 


exactly what I'm doing. 
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i THE COURT: So walk me through that again a little 


Si THE WITNESS: Sure. So in statistics you have a 

4 hypothesis, and the hypothesis is you have a null hypothesis. 
5 The hypothesis is. 

6 THE COURT: No correlation. 


7 THE WITNESS: No correlation right? And so all you 


8 could say is either you reject the hypothesis or you fail to 


9 reject the hypothesis. And the reason we use those terms is 


12:08 10 because you can't say I accept the hypothesis. That's what 
11 /SR-RS is talking about is it true. 
12 THE COURT: That there is a correlation? 


13 THE WITNESS: Not that there is a correlation but that 


14 it was random that it was traded randomly. You can only prove 


15 that it wasn't random. You can't prove that it was random. 


16 Does that make sense? You can prove that it wasn't random by 


17 comparing something that is random against the observed data, 


18 right? But you can't prove that it is random. You can only 


19 prove that it wasn't random. That's the distinction. He's 


12:09 20 going into kind of, you know, statistical nonsense here, but, 


21 yes. 


22 O.. Would you agree that /TPHAUPBs are important when you're 


23 making /( claims of one in a trillion chance? 


24 A. They are, and that's why I interpret them correctly 


25 (nuances). 
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1 : Let me show you Exhibit 2? 


2 THE COURT: By the way, in that article does it ever 


3 state it the way you stated it? 


4 THE WITNESS: There's a bigger article. It states 


5 that in one it talks about it's like the null hypothesis. If 


6 you look at one, that's talking about the fact of what a 


. p-value is. The p-value tells you the chance of seeing the 


8 observed given the null hypothesis. Can you show me the No. 1 


9 again. P-values can indicate how incompatible the data are 


£ 


12:10 10 with a specific statistical model," and that means how often 


ala what's the probability -- p-values can tell you what the 


12 probability is seeing the observed how incompatible seeing the 


13 observed is with a specific model which is, you know, the null. 


14 Q. And then it goes on to say as well "Scientific conclusions 


15 and business or policy decisions should not be based only on 


16 whether a p-value passes a specific threshold"? 


17 A. Right I view statistics as part of a mosaic of information 


18 that should be considered. Yes, a static is one piece of 


19 information that is useful, but it doesn't -- like correlation 


12:11 20 and causation aren't the same thing, so it's -—- the claims that 


21 I'm making is that the data show there is a correlation. 


22 That's the claim. ['m not explaining what that correlation is, 


23 and I think 3 is talking about you shouldn't make the decision 


24 on the static alone. You /( should see all of the evidence. 


25 THE COURT: That's what you would tell the jury? 


Case 1:21-cr-10104-PBS Document 165 Filed 01/27/23 Page 105 of 146 


105 


1 THE WITNESS: That's right. 


2 Q. But the only thing you're correspond relating is use of 


3 /TPAOEUGTS 1 and 2, correct is this? 


4 A. ight. I am describing the data. The data shows that 


5 there is a correlation, and the probability of seeing that 


6 correlation under the null hypothesis that there's no 


7 correlation is less than one in a trillion. 


8 Q. But isn't this implicitly or explicitly implying that the 


9 only reason for these transactions is the use of fighting 1 and 


12:12 10 2? 


11 , - © don't think so. Not in my opinion. (Filing agent) 


13 : Is this the contingencies table that you created? 


14 : Yes, it is. 


15 Q. And this /PHAOEFPS 38,000 trading events DFIN and Toppan 


16 on this side traded not traded on this side, whether they wer 


LS) used by DFIN and Toppan or whether DFIN and Toppan was 


18 uninvolved, correct? 


19 : That's right. 


12:12 20 ; And you get a fisher exact p-value of zero; is that right? 


21 : It's a very, very small number but it goes beyond what the 


22 output statement can produce. It's like, you know, one ina 


23 trillion is like E minus something big. 


24 Q. It's very simple these days to run a Fisher Exact Test; is 


25 that correct? 
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al A. 

2 Q. There's plenty of online calculators that you can just 
3 plug these exact figures in and get the statistic? 

4 A. That's right. 


5 QO. And this is one of those online calculators that are used 


6 to create statistics, correct? 


7 A. If you're telling me, I believe you. 
8 Q. Okay, but it matches your results, the exact statistic 
9 value is less than 0./# 00 /SR-RS 05? 


12:13 10 : That's right. 


‘lal ; And you can play around with statistics, I'm sure. 


12 : Is this you playing around with statistics? 


13 : Exactly. 


14 . Okay. 


15 , Let's assume the reverse, that Mr. Klyushin only traded in 


16 four events relating to fighting 1 and 2, didn't trade in 40 


17 events or traded this 40 events that are unrelated to fighting 


18 1 and 2, you would still get the same statistical value, 


19 COrrect? 


12:14 20 A. Yes, so to interpret this table, what you'd be saying is, 


21 the market share of DFIN and Toppan is 44 percent. They traded 


22 only in 10 percent. Is 10 percent different than 44 percent? 


23 And the answer yes, yes they're different, 44 */*. 


24 Q. So meaning the only time in essence the Fisher Exact Test 


25 would show most statistical significance is when Mr. Klyushin 
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1 only 44 percent of the time or 56 of the time traded in 


2 /TRAOEUGT 1 and fighting 2? 


3 A. No, it depends on the number of trades that he traded. So 


4 imagine a coin, right? So you flip a coin three times. It's 


5 heads all three times. How confident are you in that coin of 


6 what the mean of that coin is, right? Is it a fair coin? I 


7 don't know. Heads is pretty, you know, it comes half the time. 


Te 


8 Because you have a small sample, you don't know. you flip 


9 that coin 360 times, you're very sure that the mean is correct 


12:15 10 because you flipped the coin 350 times. 


11 Q. But you would agree that picking stocks, trading in 


12 stocks, making a decision to buy or sell, that's not flipping a 


13 coin, correct? 


14 A. Sure. The coin example is under the null hypothesis. So 


£ 


you see an observed value of 343 over 356, you 


15 you're saying i 


16 know, if you flip a coin 356 times how many times would you get 


17 343? You're pretty confident because you flipped the coin 


18 356 -- 356 is actually a pretty big number, and so you can 


19 derive a lot of confidence, the confidence interval is fairly 


12:15 20 tight. I think that's what you're getting at. 


21 Q. I'm getting at the fact that it's difficult to equate the 


22 stock market and choices of stocks and the decision to buy or 


23 sell to chance? 


24 A. I agree because people don't trade randomly. I agree with 


25 
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1 Q. They make selections maybe they like a stock maybe all of 


2 my transactions are, for example, in Tesla stock, correct? 


3 A. It's possible for a many trader to have a trading 
4 strategy, yes. 


5 Q. And in that event if I only traded Tesla stocks then all 


6 of my stocks would be in fighting 1 or 2-related stocks? 


i A. Yes if I knew what stock you traded in, I would be very 


8 confident that it was filed by DFIN and Toppan. 


9 O. It's very unlikely that -- 


12:16 10 A. But that wouldn't mean anything because they only trade 


11 this Tesla, right? It's as part of all the information, 


12 because they traded in so many different stocks, and you looked 


13 at the stocks, they traded in like, I don't know, at least -—- 


14 don't know the number of stocks that each individual traded in, 


15 but a wide variety. It's the fact that he's trading in a wide 


16 variety of stocks and they happen to all be DFIN and Toppan. 


17 You're right if it was concentrated this one it wouldn't tell 


18 you something. You could have an easy explanation. You're 


19 saying there's a trading strategy. The /SRAGT trading strategy 


12:17 20 would be we picked Tesla /SR-RS the same results that you see 


AL in the data. Does that make sense? 


22 Q. Not to me, but I'll leave it to Judge Saris. Okay, assume 


23 you trade exclusively in S&P 500 stocks, assume that 75 percent 


24 of them are serviced by device or Toppan, you make those 


25 transactions with he have */* but DFIN and Toppan only serviced 
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1 44 percent of the stocks overall. 


2 A. Right, if, you know -- I didn't do this analysis so 


3 don't want to -- you know, I don't believe that 70 percent of 


4 S&P 500s are DFIN and Toppan, but if I, you know -—-— 


5 QO. Assume it is. 


6 A. Yeah, so then you could find a trading strategy. You 


7 would say my /-D trading strategy is I only trade in S&P 500 


8 stocks, and that would be your explanation, I agree. 


9 Q. But you're looking for a trading strategy, but you would 


12:18 10 also, if you do the correlation /STA* that you did in this case 


11 you would find it would be a one in a trillion chance that it's 


12 uncorrelated to the choice of a filing agent? 


13 A. Say that again? I think you're making -- 


14 THE COURT: In other words, there could be another 


LS explanation? 


16 THE WITNESS: Yes, I agree that there could be another 


17 explanation. 


18 Q. But the only explanation you considered is fighting 1 or 


19 fighting 2? 


12:18 20 A. And after you guys submitted your report I considered 


21 market cap and sector. 


22 Q. Even though there's significantly many more variables to 


23 consider in connection with stock purchases? 


24 A. Yes. There exist trading strategies that would get you to 


25 the same result, but I don't know what his trading strategy 
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i was. What I'm describing is that whatever trading strategy he 


2 used looked like this, right? So if I knew what his trading 


3 strategy was, I could then do a better static. I don't know 


4 what his trading strategy was (static) /(. 


5 < This is Exhibit G, your second opinion, correct? 


6 : That's correct. 


7 : Once again, where did you get the data to come up with 


9 , That same dataset that I gave to you. 


12:19 10 2 You made certain exclusions /SR-RS correct? 
Ved. ‘ Yes. 


12 : What did you exclude? 


13 : What did I exclude? I made the choice to look at only 


14 earnings meets and earnings misses. I chose to try to 


LS exclude -- sorry. I /SR-RS I tried to exclude earnings meets. 


16 Q. Why is earnings meets not relevant in this analysis? 


17 A. Because your expectation for an earnings meet has less, 


18 like -- if I knew that they met earnings, I would not expect 


19 one way or another for the stock price to go on earnings alone. 


12:20 20 There are things other than earnings which determine a stock 


21 price. 


22 Q. Correct. But you only looked at earnings? 


23 A. In this I looked at earnings, yes. 


24 Q. And the only variable you considered is, from what 


25 recall, EPS? 
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2 Q. What other variables are there in connection with 
3 earnings? 


4 A. There's a lot. Would you like me to list them. 


5 OQ. A couple, if you remember them. 


6 A. There's earnings guidance, revenue. But these aren't 


£ 


for every cop, */* company, so not every company 


7 reported 


8 issues guidance but every company issues earnings. 
9 : And not every company has /SR-RS? 
12:21 10 ; That's true. 


11 ; So /SR-RS your dataset isn't complete? 


12 : I think for all but one of his -—- once he traded, all but 


13 I think I had earnings for? 


14 THE COURT: Had earnings —-— 


15 THE WITNESS: Earnings sometimes and earnings actuals 


16 . He's saying that I don't have earnings for everything, 


£ 


17 I do have earnings for everything in this universe. 


18 : Almost everything? 


19 : Except for I think -—- you could look at it. I don't know. 


12:21 20 I don't know exactly. I'm pretty sure it's one. It could be 


21 three. It wouldn't change my results. 


22 Q. And what is the null hypothesis that you created for this 
23 test? 


24 A. That trading long or short was unrelated to the earnings 


25 surprise, whether they meet it by a lot or whether they miss by 


12:22 10 


12:23 20 


21 


22 


23 


24 


25 
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Q. So you expect to see essentially a 50/50 distribution by 


buying and selling? 


A. I expected to see about 6040 because companies tend to 


beat earnings as opposed to miss earnings. 


So 6040 and you saw here 86 and 14? 


Yes, 86 plus -—- yes, yes. 


And again 


teams analysts 


you would agree that there's industry industries 


working on predicting earnings better than other 


analysts or what the market expects? 


A. I agree. 
Q. Would you 


correctly predi 


agree that a good analyst would be able to 


ict 86 percent of the time whether there's an 


earnings surprise or an earnings miss? 


A. I don't know the answer to that, but, yeah. 


Q. What was the third statistic that you looked at? 


A. It's called a permutation test. 


O% Permutation test and this was looking at device inn 


data, correct? 


A. That's right. 


Q. And this is what you came up with the scatter plot 


essentially? 


A. 


Q. And this scatter plot /SR-RS are number of hours? 


A. Prior to earnings, yes. 
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1 Q. Is it fair to say based on this plot that most the vast 
2 majority of transactions of these traders initiated happened 


3 probably the day before? 


4 A. I'd say most. 


5 Q. Most. But Mr. Klyushin potentially */* for Mr. Klyushin, 


6 potentially even more? 


8 ‘ And that's consistent with your understanding of earnings 


9 EUGDZ? 


12:23 10 


11 Q: You would essentially by the day before see wait until 


12 the earnings came out and then make a decision to sell the 
13 stock? 

14 A. Yeah. You want to limit your exposure to 

15 nonearnings-related stock pricing events. 


16 Q. And in connection with the intrusions the vast majority of 


17 them took place at least 30 to 90 hours before that? 


18 A. It looks to me about a quarter happened in each -- I could 


19 calculate it. I don't know. The vast majority is certainly 


12:24 20 wrong, but 75 percent? 
21 7 75 percent is this? 
22 : Sure. 
23 : A good majority? 
24 : YES. 


25 : So isn't this inconsistent with the notion that -- strike 
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1 that. The fact that almost all of the intrusions took place 


2 more than 30 hours before an earnings announcement, that the 


3 trading was consistent Mr. Klyushin with other traders who 


4 trade earnings by trading the day before, aren't you just 


5 walking in or expecting the conclusion that obviously the 


6 earnings transactions or the transactions the /SR-RS sells were 
7 after the intrusion? 


8 A. So I think that would be the case if all of the dots were 


9 within the ten hours. I don't know if you can —- but if all 


12:25 10 the dots were in your top circle, I think that's true. But 


11 let's look at the bottom left, right is this so here are cases, 


12 you know, I counted off one, two, three, four, five, six, 


13 seven, seven cases where a download occurs three days early, 


14 yet they never or they only traded one time after sorry -- 


15 before the download. So your expectation would be that 


16 25 percent should be in that little cloud at the bottom left. 


Ay Yet it looks like, you know, it's 90 percent right if it's nine 


18 dots out of ten. Does that make sense is this. 


19 THE COURT: No. Could you walk me through this again. 


12:26 20 All right the start of trading -—- 


21 THE WITNESS: Sure they often don't trade early, but 


22 when they trade early it's 90 percent after a download. 
23 THE COURT: And by a download you mean the hack is 
24 


25 [TNESS: The hack. It's a download from the 
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1 server, yes. 


2 THE COURT: A hack? 


3 THE WITNESS: Yeah they like, you know, I think this 


4 chart shows that. 


5 The only time they start trading early is after there 


6 is a hack that's early. 
7 THE COURT: And which blue dots show that? 


8 THE WITNESS: The bottom left near the minus 80, minus 


12:27 10 THE COURT: So there are only a few of those? 


11 THE WITNESS: That's true, yes. But it's really the 


12 whole diagonal, you know? 


13 Q. I the vast majority of trading, even though they had -- 


14 A. ight the question is that clearly a pattern emerges, 


15 right is this when I caught this the expectation there should 


16 be no pattern that emergeses. We're talking about someone 


Ly downloading for whatever reason on the server and when people 


18 start to trade, right, your expectation there should be no 


19 pattern should emerge. But when I plot it, I see a pattern 


12:27 20 that occurs. And then I say yeah you're right there's only 9 


21 dots. Let's use statistics. Statistics can tell us how often 


22 this would occur by chance alone, right? And I find that in 


23 this case this type of pattern where all of them /SR-RS that 


24 little triangle, it only occurs one in a million times. 


25 Q. But then again the fact that almost all of the trading 
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1 took place the day before which is consistent with how other 


2 traders trade earning reports, you're essentially /SR-RS? 


3 A. I don't /SR-RS I include it in my statistic but you're 


4 right those trades have less statistical power right but the 


5 trades that have more statistical power I run the stat to be 
6 able to control for the issue that you're talking about. 
7 Q. So more statistical power meaning one this a million is 


8 not as statistically significant? 


9 A. No. Meaning that the pattern -- you're right the pattern 


12:28 10 can't emerge if they treated that 359, 359 they would always 


11 trade after. There would be some like that just would be 


12 the case, and the statistic would show that there's no 


13 correlation. But because they don't trade like that, they 


14 sometimes trade early, I can then run a statistic that shows -—- 


15 I can see this pattern emerging and I can see how likely that 


16 pattern Stow merge. 


ay Q. But the pattern here is likely. It doesn't matter, for 


18 example, when it was accessed if it was accessed 90 hours 


19 before they still trade only 10 hours before, correct? 


12:29 20 A. Sometimes but not all the time. 


21 Q. A lot of the time, though, right? At least half of the 


22 trading was —- 


23 A. What's striking is that there are like a little lots a lot 


24 of dots. There's a lot of excite space, right? So when they 


25 trade early, there's downloads that are early. 
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1 Q. But there's also when they trade early downloads that are 
2 very, very -- 

3 A. Far away, that's true. 

4 Q. 60 hours, 40 hours, 90 hours, 80 hours. A significant 

5 portion of the downloads meaning it's not like an intrusion 

6 occurred, a trade was placed. This could be an intrusion 

- occurred, and a trade was placed many days later? 

8 A. That's right. 

9 Q. Okay, but you're saying that there's only one in a million 


12:30 10 chance that Mr. Klyushin at random would place these trades? 


1] In connection with the download -- 


12 A. That this pattern would emerge if they were uncorrelated 


13 only less than one in a million times. But do you see a 


14 pattern? I see a pattern. 


15 Q. Yes, I see a pattern that the vast majority of the trades 


16 occurred less than ten hours before the earnings event? 


17 A. Right but do you see the other dots the low dot line. 


18 QO. And almost all of the download occurred more than 


19 24 hours, 30 hours. 


12:31 20 A. Sorry, owe did you want me to respond to that. 


21 Q. That's the pattern I see. Do you see any other pattern? 


22 A. I see the pattern that when the downloads occur early -- 


23 they sometimes trade early but they only trade early when the 


24 download occurs early. That's what I see. 


25 Q. You also made certain exclusions of data, correct, when 


12:31 10 


12:32 20 


21 


22 


23 


24 


25 
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A. What do you mean by 


Enc? 


that? 


You excluded any trading this 2017, for example 


I wasn't asked to look at that time period. 


So isn't it relevant 
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7 


that there were transactions in these 


same exact stocks prior to any download? 


levant to what? 


Relevant to whether they traded -- their trades are 


correlated to a download. 


I think I answered this before, which is that i 


1fferent analysis, right? So I would say is the 


that time period related to DFIN and Toppan. 


But you also said more data is good. 


Uhm, more data is good, yes. But that's not th 


that I was asked to look at. Actually I didn't even 


before -- I don't think 


starting in October of 20 


have. 


had much of this data. So 


17, so anything before that 


Q. You didn't pull all of the transactions for the 


prior to 2018? 


A. There was only one account that I had that was 


and the trading data that 


THE COURT: Let me ask you, 


had began in October of 


EV: Five minutes, your Honor? 


t would be 


trading 


e guestion 
have data 


had data 


didn't 


se traders 


in Sladkov, 


2017. 


how much longer do you 
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1 THE COURT: I just want to make sure. Will you have 


Si MR. FRANK: Redirect? Super brief, your Honor. 


4 THE COURT: Redirect? All right. I just want to make 


5 sure we finish this morning. 


6 Q. And you also excluded any trading after December 2020 you 


7 excluded trading in December 2021 even though there's no 


8 evidence of downloads during those time periods, correct? 


9 A. I don't know of the evidence of the downloads. 


12:33 10 Ou It wasn't on the list of device inn downloads that you 


11 reviewed, correct? 


12 A. I don't -- I don't think -—- yeah, I don't remember the 


13 time period for the device inn downloads standing here today. 


14 I can look at it. 


iS) . NEMTSEV: have nothing further, your Honor. 


16 RE ECT EXAMINATION BY MR. FRANK: 


LS] O% If I could just pick up where Mr. Nemtsev left off, he 


18 asked you some questions about these trades in 2017. Do you 


19 recall that? 


12:33 20 A. 


21 Or And he asked you whether you looked at their trading, 


22 plural, in 2017. Do you recall that? 


23 A. 
24 Q. Are you aware of whether Mr. Klyushin had a trading 


25 account in 2017? 
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1 A. To my knowledge, his trading account began in 2018. 


2 Q. And of the trading accounts that you looked at, you looked 
3 at eight traders? 
4 : That's right. 
5 : How many traders even had a trading account in 2017? 
6 : Just one. 
7 ; Who was that? 
8 : Sladkov. 
9 : So these are Mr. Sladkov's trades? 
12:34 10 : That's correct. 


11 ‘ Are you aware that Mr. Klyushin claims he doesn't know 


12 . Sladkov and has no relationship with him? 


13 : I was not aware of that. 

14 Q. Would it make sense to analyze the trading of eight 
15 traders when only one of them is trading? 

16 A. Can you repeat the question? 


17 Q. Would it make sense to analyze the trading 


18 traders when only one of them is trading? 


19 A. Uhm, it would answer a different question, like —- 


12:35 20 Q. Could we look at Exhibit K, please. And I think for this 


21 we need to switch back to Here, we'll do it on paper. 


22 Mr. Nemtsev asked you some guestions about this chart. Do you 


23 recall those questions? 


24 A. Yes. 


25 Q. So as we move down the vertical axis, that is trading that 
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1 gets further and further away from the earnings announcement, 
2 correct? 


3 A. That's right. 


4 Q. And if we move left across the horizontal axis, that is a 


5 download that occurs further and further away from the earnings 
6 announcements, correct? 
7 A. Correct. 


8 Q. So there are in fact trades that occur 70, 80, 60, 50, 


9 40 hours before an earnings announcement, correct? 


12:36 10 A. That's right. 


11 Ox In each and every one of those instances, when was the 
12 download? 


13 A. Before the trade. 


14 . Why is this part white? 


15 : Because it never happened that they traded before the 


16 download. 


ay Q. He asked you guestions about the fact that a lot of 


18 trades are concentrated near the time of the earnings 


19 announcements, sort of up here at the top of the chart? 


12:36 20 A. Uh-huh. 


21 QO. In each and every one of these instances, there was 
22 download before the trade, correct? 


23 A. That's correct. 


24 Q. Tf a download had happened after a trade, where would that 


25 be on the chart? 


12:37 10 


12:38 20 


21 


22 


23 


24 


25 
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The trading after? It would be in the red triangle. 


So if he traded near an earnings announcement but the 


load was also near an earnings announcement, but the 


load was afterwards, it would be in this area, correct? 


That's right. 


But there's nothing in this area? 


That's right. 


So what is 


data? 


the correlation that you observed based on the 


A. For Klyushin, I observed that the probability of seeing 


this pattern emerge, if the two things were uncorrelated, is 17 


in a million. 


Q. Okay. So the null hypothesis is that if there's no 


correlation between a download and a trade, this pattern would 


occur 17 times out of a million? 


Yes. 
And you're 


testifying 


not explaining why the pattern emerges? You're 


that there is a correlation, correct? 


That's right. 


Could we look at Exhibit G, please. What is the 


correlation tha 


A. That they 


t you observed here? 


tended to -- it was between them trading long 


and short and the earnings surprise being positive or negative. 


QO. Okay. And 


by definition, an earnings surprise is 


unexpected by the market, correct? 
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1 A. That's why it's called a surprise. 


2 Q. Okay. 86 percent of the time when the surprise was 
3 positive, Mr. Klyushin bought stock, correct? 
4 A. When it turned out to be positive, yes. 


5 Q. And 85 percent of the time when the surprise was negative, 


6 Mr. Klyushin went short, correct? 


7 A. That is correct. 


8 ‘ Could that be the result of remarkable skill? 
9 . Yes. 
12:38 10 : Could that be something else? 


Ad. ‘ Yes. 


12 ‘ Are you opining what the reason for that is? 
13 : No. 
14 : You were asked some questions about Defense Exhibit 4 and 


15 1 statement that "P-values do not measure the probability 


16 the studied hypothesis is true or the probability that the 
ay were produced by random chance alone." 


18 : That's right. 


19 : In any of the analysis that you did, are you opining that 
12:39 20 the null hypothesis is true? 
21 : No. 


22 : What are you opining? 


23 : I'm opining that the null hypothesis is false. 


24 ‘ That the null hypothesis is false. So the hypothesis in 


25 every one of those instances that we've looked at is that 
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al there's no relationship between X and Y, correct? 
2 A. 


3 Q. And you're opining that that hypothesis that there is no 


4 relationship is false, correct? 


6 ‘ Because there is a statistically significant relationship 


7 between those two things? 


9 : Are you suggesting that Mr. Klyushin traded by chance? 


12:40 10 


11 ; What are you suggesting about the relationship between 


12 . Klyushin's trades and who the filing agent is? 


13 : I find that he trades in DFIN and Toppan Merrill earnings 
14 events a lot, and you wouldn't see that, given the market, the 


15 market share. 


16 Q. So the null hypothesis that there is no relationship 


17 between his trading choices and the filing agent is false? 


18 A. Right. You'd have information. The correlation doesn't 


19 equal causation. 


12:41 20 Q. So there is a statistically significant relationship 


21 between how he trades and who the filing agent is? 
22 A. 


23 Q. Are you offering an opinion on what the trading strategy 


24 is that results in that correlation? 


25 A. No. 


12:42 10 


12:42 20 


21 


22 


23 


24 
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Are you opining on what the cause of that correlation is? 


No. 


You're simply testifying that there is a correlation? 


That's right. 


And what is the correlation, according to the Fisher Exact 


Test? 


A. That you would see those occurring by chance less than one 


in a trillion times, 


yeah. 


QO. And the Fisher 


for the first two tests, and the rest -- 


Exact Test calculates all the different 


possible combinations, correct? 


Yes. That's why the number gets so big or so small. 


Why is it called an "exact test"? 


Because it's calculating an exact probability. 


Are you making distributional assumptions in coming up 


that number? 


No. 


It's simply counting all the different combinations and 


coming up with the result? 


A. That's right. 


MR. FRANK: 


THE 


that someone 


doctorate or 


THE 


COURT: 


A computer does it. 


No further questions. 


One quick question. When you said before 


checks your work, is that somebody who has a 


a master's in statistics? 


W 


ITNESS: 


Sure. So there are two layers of 
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1 checks. There are conceptual checks and then there's data 


2 checks. I had someone who doesn't have a degree check my data 


3 work, and then I had someone with a degree check my conceptual 


4 work. 


5 THE COURT: And those are employees at the SI 


6 THE WITNESS: That's right. 


7 THE COURT: And do you know what the degree is? 


8 THE WITNESS: Ph.D. in finance. 


9 THE COURT: What is it? 


12:43 10 THE WITNESS: Ph.D. in 


1] BY MR. FRANK: 


12 Q. But leaving aside degrees you obtained in 201 


13 many years have you been working with statistics i 


14 day-to-day work? 


15 A. Since I've graduated, I worked with -—- this is my job. 


16 I do it all the time. 


ay Q. lve years? 


18 A. Twelve years, yes. I have significant practical 


19 experience in matters of import to the Commission, yes. 


12:43 20 THE COURT: Thank you. 


21 BY MR. FRANK: 


22 QO. And the defense expert, Mr. Cullen, he was in college in 


23 2012 to 2016. Are you aware of that? 


24 A. I was not aware of that. 


25 Q. Where were you in 2012 to 2016? 


12:44 10 


12:44 20 


21 


22 


23 


24 


25 
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I was working at Cornerstone 


Doing statistical 


And in 2016 and 2019, 


degree. Are you aware of 


MR. F 


KERN 


CH: 


analysis? 


127 


he was studying for his master's 


that? 


Objection, foundation. He already said 


he wasn't aware of it. 


Where were you between 2016 and 2019? 


Doing what? 


I was working at the SEC. 


Summarizing data and drawing conclusions from that data 


using statistics. 


MR. FRANK: 


THE COURT: 


MR. N 


EMTS 


EV: 


THE COURT: 


THE W 


ITNE 


SS: 


Thank 


Thank 


No, 


Thank 


yo 


yo 


yo 


yo 


u. 


No further questions. 


Anything else? 


ur Honor. 


You may step down. Thank you. 


Thank you. 


(Witness excused.) 


THE COURT: 


MR. Fernich: 


THE COURT: 


course, but 


witness on. 


So what do we do now? 


How about the bathroom? 


I want to know -- it will come in due 


just want to know whether you want to put your 


I mea 


n, 


MR. N 


one second? 


EMTS 


EV: 


not today obviously. He's sick. 


Can 


con 


fer with my colleague for just 


12:45 10 
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Yes. 


(Discussion 


MR. FERNICH: 


off the record between defense counsel.) 


Judge, if you're prepared to hear 


argument today -- you indicated last time that you'd probably 


be ruling from the bench -—- maybe we should just proceed with 


the argument and see 


THE COURT: 


MR. FRANK: 
THE COURT: 
the argument, and if 


witness. 


MR. FERNICH: 


we're suggesting. 


how it goes. I'm not reserving -- 


Let me ask you this: Does the government 


Yes, your Honor. 


Now it's your turn. I'm not going to do 


you don't win, then you get to put ona 


No, no, your Honor. That's not what 


MR. NEMTSEV: 


would just ask your Honor to admit 


Exhibits 1 through 5 


12:45 20 


21 


22 


23 


24 


25 


THE COURT: 


THE COURT: 


and Exhibit 7 that we used. 


Yes, okay. 


(Defense Exhibits 1-5 and 7 received in evidence.) 


Why don't we do this. It's quarter to 


1:00. We spent the morning on this. There's at least been a 


humanitarian request 


finish argument by 1: 


for a break. I don't know that we would 


00 o'clock, so I need a game plan from you 


all. Why don't we do this: Why don't we take a five-minute 


break right now. We'll come back in, and you'll tell me what 


you want to do. We also have to do a pretrial at some point; 
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i for example, voir dire questions for the jury, and there may be 


2 other pieces of questions on motions in limine I haven't 


3 addressed. I'm not sure. But I want to know now because 


4 don't want to -- I'm gone next Thursday and Friday, so I need 


5 to schedule something if you're going to want to have another 


6 witness. 


7 MR. FERNICH: We're also sensitive to the Court not 


8 being so well today, and maybe you don't want to hear argument 
9 today, so, you know —--— 


12:46 10 THE COURT: I'm here. 


11 MR. FERNICH: Okay. 


12 MR. FRANK: I'll just say for the government, we don't 


13 anticipate a long argument, Judge. We presented the evidence; 


14 we'll make a very brief argument. 


15 THE COURT: Why don't you do this: Why don't you 


16 confer with your client, take a bathroom break, and I'll see 
17 you in five minutes, all right? 


18 MR. FRANK: Yes, your Honor. 


19 (A recess was taken, 12:46 p.m.) 
12:49 20 (Resumed, 12:57 p.m.) 


21 THE CLERK: So as the proponent of the testimony, the 


22 government would go first. 


23 MR. FRANK: Thank you, your Honor. So your Honor 


24 identified a couple of series of issues that you were focused 


25 on for today. One was the qualifications of the witness. The 
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1 relevance of the testimony was the second. The reliability of 
2 the testimony was the third, and avoiding the prosecutor's 

3 fallacy was the fourth. So let me just address those briefly. 
4 With respect to the witness's qualifications, he has 


5 studied statistics. He graduated with a degree in economics 


6 and mathematics that required the study of statistics more than 


a a dozen years ago. He is a chartered financial analyst, which 


8 is an exam that requires three layers of testing that examines 


£ 


9 his quali 


ications in statistics. But I would say most 


12:58 10 importantly, your Honor, he has spent the last dozen years 


11 working at blue-chip institutions where his work has been 


12 focused on conducting statistical analysis in the real world. 


13 Cornerstone Research, where he spent the first three years 


14 after his graduation, is the blue-chip consulting firm used in 


15 litigation; and his work there for banks and financial 


£ 


16 institutions and pension funds was focused on the use of the 


£ 


or the various cases that he 


17 appropriate statistical analysis 


18 was working on. And since that time, he has been promoted 


19 through the SEC. He has won awards within the SEC for his use 


12:59 20 of some of the very statistical methods and analysis that he is 


21 applying in this case. His work has been checked and rechecked 


22 by others at the SEC, and there's really no meaningful dispute 


23 that he is not qualified to offer the opinions he's offering 


£ 


24 here because of the fact that he has done this for a dozen 


25 years. It's not dissimilar in the slightest from a witness 
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al from the FBI ballistics lab testifying about the type of work 


2 that they do there on a day-to-day basis to analyze ballistics 


3 for fingerprints on a weapon. This is what he does, and it is 


4 what he has done, and it is what he's trained to do. 


5 With respect to the relevance of the testimony, your 


6 Honor, the testimony is extremely relevant and probative of 


7 what's at issue in this case. First of all, he looked at the 


8 relevant time period, which is to say the time period of the 


9 charged conspiracy, and examined the correlation between the 
01:00 10 defendant's trading and the nature of that trading and what is 


11 at issue in this case: 


12 Is there a correlation between the trading and who the 


13 filing agent is? The answer is, there is a correlation. 


14 Is there a correlation between the trading around 


LS earnings and the degree of earnings surprises, which is the 


16 nature of the trading that is at issue in this case? There is 


1] a correlation. 


18 Is there a correlation between the defendant's trading 


19 and things that were downloaded from Filing Agent 2? There is 


01:00 20 a correlation. So it is highly relevant to the issues here. 


21 And it is also relevant and probative for another 


22 reason, your Honor. The defense has found examples, they will 


£ 


23 continue to find examples, they will offer in evidence examples 


24 of individual trades that went a certain way or didn't goa 


25 certain way, and the government will also offer by way of 
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1 example evidence of individual trades. But there are thousands 


2 of trades at issue in this case across eight different traders 


3 and, you know, roughly two dozen trading accounts over a 


4 roughly 31-month period, right? If we were to go through each 


is and every one of those trades, and each and every one of those 


6 earnings announcements, and each and every one of the hacks 


7 that took that information, and all of the other possible bases 


8 for making that trade, we would literally be here for years, 


9 months, analyzing each one of those things. 


01:01 10 Similarly, if we were to only go by example, it would 


11 be highly misleading to the jury because they're going to find 


12 examples that are within the 15 percent where he traded 


13 opposite the way you would expect based on the earnings, and 


14 we're going to find examples within 86 percent of the time 


15 where he traded consistent with the direction you would expect 


16 based on the earnings announcement. So the examples, while 


17 Significant, only get you so far. What the statistical 


18 analysis does is, it shows which are the outliers and which are 


19 consistent with the bulk of the trading that went on here, and 


01:02 20 is there a statistically significant correlation? That's why 


21 it's probative, Judge. 


22 And it's reliable, it's reliable because he's using 


23 standard tests. He's using off-the-shelf software to conduct 


24 that analysis. He's using the relevant dataset, which is the 


25 trading by these traders during this time period across these 
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1 accounts, and it's been checked and it's been rechecked. And 


2 there's really no meaningful -- I think after that 
3 cross-examination, what the defense may establish is that they 
4 can ask some questions about some of these things and they can 


5 argue to the jury about the weight they should give this 


6 evidence, but there's really no dispute that the statistical 


7 Significance of the relationships that he's established exist. 


8 They exist. That correlation exists. It is statistically 
9 significant. There's really been no contrary suggestion. 


01:03 10 And that leads us to the last point, which is the 


11 concern about the prosecutor's fallacy. I submit that this 


12 witness was extremely careful, your Honor. 


13 THE COURT: Yes, the one thing on that, I might agree 


14 with that, and I understand it a whole lot better, but I don't 


15 want you to mention that number in the opening statement. 


16 MR. FRANK: That's fine, your Honor. 


17 THE COURT: Because it needs to be very, very, if it 


18 comes in, very nuanced where someone understands there's a 


19 difference between correlate and cause, and that there are 


01:04 20 other explanations, like, he likes a certain kind of stock. 


21 MR. FRANK: That's exactly right. 


22 THE COURT: It's got to be tailored to the Supreme 


£ 


23 Court discussion of prosecutor's fallacy, but also cabined in a 


24 way that the jury doesn't start of 


in an opening statement 


25 without knowing anything, and all they can think about is one 
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1 in a trillion. 


2 MR. FRANK: Understood, your Honor, and we will do 


3 that, and we will be extremely careful in the way we present 


4 his testimony. But the point is that he is extremely careful 


5 in how he frames it. He was asked many, many times to offer an 
6 opinion on causation. He is one piece of the puzzle. He is 


7 not opining about causation. He's simply showing the 


8 relationship, and we are then going to offer through other 


9 evidence an explanation of the -—- 


01:04 10 THE COURT: This is foreign turf for most jurors. 


ala don't know if you've run into this at all. Have you been in a 


12 trial where the jury could ask questions if they don't 


13 understand the numbers? 


14 MR. FRANK: I have had that experience twice, your 


15 It was not my favorite experience, but it has happened. 


16 THE COURT: I'll ask you all what you think. All 


17 right, thank you. 


18 From the defense point of view, as I understand it, 


19 can't remember if we were on the record or off the record, but 


01:05 20 you do not want to put on your expert right now? 


21 MR. FERNICH: That is correct, Judge. We're prepared 


22 to argue it and submit the issue today. 


23 THE COURT: You're prepared to -- 


24 MR. FERNICH: To argue and submit the issue today. 


25 THE COURT: Oh, all right. Thank you. 


Case 1:21-cr-10104-PBS Document 165 Filed 01/27/23 Page 135 of 146 


135 


ac MR. FERNICH: And fundamentally, I'm honing in on what 


2 your Honor just said. The argument is based upon the 


3 prosecutor's fallacy, and we're moving to exclude what they're 


4 calling the p-value, and what I've learned in other cases is 


5 called the probability ratio, and it's very simple: We start 


6 with the proposition that under Rule 702, expert testimony is 


7 inadmissible if it merely tells the jury what result to reach. 


8 That's from the commentary to Rule 702 and the First Circuit 


9 case from 1995 called U.S. v. Shea. 


01:06 10 THE COURT: I agree with that, but that's why I was 


11 very careful that we're not going to do it in the openings, and 


12 we're going to limit what he can say, just like you nicely did 
13 on cross. 
14 MR. FERNICH: That's right, and the cross was 


15 instructive and the redirect was instructive, in my view, based 


16 on two things he said. In response to a question from 


17 Mr. Nemtsev, he said, quote —-- and this was about results 


18 having practical significance when Mr. Nemtsev showed him the 


19 article -- and he said, quite rightly, "96 percent being 


01:06 20 different from 44 percent is of practical significance." 


21 I agree with that proposition. And when you look at 


22 the chart in Exhibit G, there are six of eight categories 


23 there, and everything is fine in terms of admissibility in 


24 categories in Lines 1 through 6. When he goes the extra yard 


25 and he gets close to the prosecutor's fallacy with the p-value, 
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1 or the probability ratio, they don't need that. It smacks the 


2 jury right across the face that 96 percent is different from 


3 44 percent. And that's of practical significance. It's not 


4 beyond the canon of an average juror. But when you start in 


5 and don't extrapolate out, piling multiplier upon multiplier 


6 and you get to one in three trillion, that's not expert 


7 testimony anymore. That's just telling the jury what result to 


8 reach. 


9 And similarly, in response to a question from 


01:07 10 Mr. Frank on redirect, he said in plain English what his 


11 analysis means, that Mr. Klyushin trades in DFIN a lot, and you 


12 wouldn't expect to see that much based upon its market share. 


13 That is fine. That's what he did in reviewing the data and the 


14 analytics. He serves in essence as a summary witness in that 


15 respect. But to then pyramid out to 1.3 trillion, that is 


16 antithetical to the state of Rule 702. It's just telling them 


17 what result to reach, and the problem is exacerbated 


18 THE COURT: The problem I run into is, just going 


19 through the check list, he does have the qualifications. 


01:08 20 MR. FERNICH: Sure. 


AL THE COURT: It is relevant. The dataset is reliable, 


22 and no one has invalidated or suggested that the Fisher Exact 


23 Test is an unreliable test. No one has put in any evidence. 


24 was sort of thinking I would hear that, but I haven't heard 


25 that. And so really what you're talking about is somewhat more 
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1 403 analysis, which is, you think everything should come in 


2 other than one in a trillion. 


3 MR. FERNICH: Well, I mean -- 


4 THE COURT: Not that you're conceding it, but that 


5 there's enough under Daubert to put everything in other than 


6 one in a trillion. 


7 MR. FERNICH: Yes. Presumably there will be a battle 


8 of the experts. We don't want to gut their right to present 


9 the case as they see fit, but the jury when it's told of that 


01:09 10 number, as we flagged, you know, much earlier in the hearings 


‘lal before your Honor, they don't need that. When I see 96 versus 


12 the 44 percent market share, it's exactly what Mr. Frank is 


13 Saying: Yes, it could be based upon, you know, hacked 


14 information. It could be based upon a contrary trading 


15 strategy. It could be any number of things. There's no reason 


16 that a putative expert, and he will be qualified as an expert, 


17 but an expert who works for the prosecuting authority in 


18 litigation and analytics, should go the extra yard and say, 


19 "Hey, it's one in three trillion that this could happen -—-" 


01:09 20 THE COURT: One in a trillion? No? 


21 MR. FERNICH: Well, there's two —- look, there's three 


22 stats, Judge. 


23 MR. FRANK: It's one in a trillion is what he would 


24 say, and -—- 


25 MR. FE : One in a trillion. There's three stats: 
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i the correlation between filing agent and trading activity. 


2 Then there's the earnings stat. There's a correct up or down 


3 position, which is I think the most problematic one because, as 


4 your Honor's questions indicated, it gets you into bona fide 


5 trading strategies, and he acknowledged that it could be the 


6 result of bona fide trading strategies, so you get into a 401, 
7 403, 702 balance there. 


8 And then the third, the timing. Number two is the 


9 most problematic, but I know what we can do and what we can't 
01:10 10 do, and what we can plausibly argue for is to get rid of the 
ala mind-boggling, eye-popping number, and let him put his 


12 underlying analysis with the 96 percent versus the 44 percent 


13 market share right before the jury. They're not idiots. 


14 They're going to get this. But if you go the extra mile -—- 


15 THE COURT: All right, so this is what I'm going to 


16 do: I'm going to think about the one in a trillion number. It 


LS) sure isn't coming in in the opening. Other than that, I mean, 


£ 


18 that would be a 403 analysis. From the point of view o 


19 Daubert, it meets the requirements, so I am worried. 


01:11 20 And let me look at the government and just say, I sort 


21 of take the defense's point a little bit. You know, be careful 


£ 


TOLrs 


22 what you wish 


23 MR. FRANK: I understand, your Honor, and if I may 


24 fly just address that. First of all, we don't need that 


25 ation. That's not the test. 
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i THE COURT: No, it is not, but it is eye-popping. 


2 MR. FRANK: But it would be eye-popping if we were 


3 saying it's a one in a trillion chance he did it, he didn't do 


4 it, or he did do it, whatever the formulation is. But that is 


5 not the testimony in this case. The testimony is extremely 


6 narrow. And what he is saying is, the probability that you 


a would observe this trading pattern, this relationship between 


8 trading and the identity of the filing agent, that is something 


9 you would not expect to see more than one in a trillion times. 


01:12 10 But not that he did it for some nefarious reason. There may be 


11 a legitimate 


12 THE COURT: 1 let it in, we're just going to have 


13 to be really careful about how it's said —- 


14 MR. FRANK: Agreed. 


15 THE COURT: -- as well as cautionary instructions. 


16 I'm struggling with it a little bit, but I understand it now. 


17 That's a big plus because I didn't when this first came in the 


18 front door, so I'm trying to treat it very gingerly. 


19 MR. FRANK: And we think a limiting instruction to the 


01:12 20 t that, you know, this is one piece of evidence that is 


21 ted at establishing a statistical relationship but not 


22 tion —— 


23 THE COURT: Is he your first witness? 


24 MR. FRANK: No. No, he's not our first witness, and 


25 he's not even our second witness. 
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1 THE COURT: All right, let me think about that piece 


2 it. I don't think it will really affect -- 


3 MR. FERNICH: Let me just make one thing clear, okay? 


4 I want to make clear that we object as a threshold matter under 


5 Rule 702, that that branch of his testimony is not admissible 


6 expert testimony because it simply tells the jury what result 


- to reach. I understand Mr. Frank's point. I understand what 


8 he's purporting to say and what he's not purporting to say. 


9 Irrespective of that, it does not meet the standard for 


01:13 10 admissibility under 702 because that branch of the testimony is 


11 simply telling them what result to reach, and they can deduce 


12 all of that from everything else, the first six columns of 


13 the chart. 


14 THE COURT: All right, thank you. We're done. But 


15 let me say, we do need another day to talk about voir dire, 


16 jury voir dire, et cetera. 


17 Do you have another date in there? 


18 THE CLERK: No, we don't. We're just going right to 


01:13 20 THE COURT: Well, we can't go right into trial because 


21 2:15. Let me just say that looking at the voir dire, I am 


22 thinking now, turning to the government, that we need to knock 


23 out the Russian Federation from the website because otherwis 


24 have to ask about it. You can just black it out, that he does 


25 work for the Russian Federation. Either that or I have to ask 


01:14 10 


01:14 20 


21 


22 
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a guestion about it. 


MR. FRANK: 


about it, Judge, 


THE COURT: 


MR. FRANK: 


no dispute. 


He's in Moscow. 


We have no objection to asking a question 


As soon as 


but we're not going to say the name. 


do -- 


He's Russian. I mean, there's going to be 


But the fact that he works with 


and for the government is going to come in in other ways. He 


says it. He says it in the chat -- 


THE COURT: 


the war in Ukraine. 


don't want to get into, 


MR. FRANK: 


THE COURT: 


Federation, 


It may be. 


don't want to get into Mr. Putin. 


just don't want to get into 


don't know -- 


We're not going near those things, Judge. 


don't know why you need the Russian 


which makes it sound like the KGB. Why don't you 


just put in that he works for the government? 


MR. FRANK: 


That's exactly what we would say. I just 


don't want to redact his website. 


more questions. 


THE COURT: 


Would you li 


MR. 


N 


EMTS 


EV: 


would al 


think that raises even 


ke the website redacted? 


so like the guestion asked 


because the website features not only the Russian Federation. 


23 


24 


25 


It also features the 


within the Kremlin, 


fact that he's working for some office 


from my recollection. It's extensive. It 


talks about specific offices —- 


THE COURT: 


So you just would prefer the question? 
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i MR. NEMTSEV: think I would prefer the question -—- 


2 THE COURT: Because you think it will come out anyway? 


3 MR. NEMTSEV: think it will. That's the issue. 


4 think at a certain point, there is some chat, whatever it may 
5 be -- 


6 THE COURT: Okay, you know the evidence better than 


7 do, so -—- 


8 MR. KOSTO: Your Honor, the one thing in terms of 


9 timing that I think is important to think about is, we would 


01:15 10 submit that the Court can handle these questions orally as part 


1] of voir dire, and then adding a written questionnaire to the 


12 jurors that will require them to sit down and fill out will 
13 burn a day that we don't have. 


14 THE COURT: Can I tell you, I'm really good at this. 


LS I've been doing it for 30 years, 30 as in 3-0. I'm not going 


16 to do a massive 30-page questionnaire. I'm going to ask one or 


17 two questions. It's, like, an opportunity for some juror who 


18 has a problem and is embarrassed to tell it to me in front of 


19 the defendant to say something. So why don't you work on one 
01:15 20 or two questions and 
21 MR. FRANK: Judge, we really object to that because 
22 that singles out this issue, and -- 


23 THE COURT: It's a big issue. Right now in our 


24 history and the history of Russia, it's a huge issue. 


25 MR. FRANK: And it can easily be addressed in the 
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1 context of a question that wouldn't embarrass anybody. 


2 THE COURT: I understand that, but we'll do it on that 


Si issue if you want it. Do you want it? I'm not going into the 


4 war in Ukraine. It's not happening. 


5 MR. FERNICH: No, no. 


6 THE COURT: I'm just going to ask whether somebody can 


7 be fair and impartial with respect to someone who is a citizen 


8 of Russia, and we'll leave it there. Thank you. I do it with 


9 black people. I do it with child porn cases. I do it with 


01:16 10 anything that's sensitive where someone might be embarrassed to 
11 raise their hand, and it will take ten seconds. No, that's a 


12 lie. Ten minutes. 


13 MR. NEMTSEV: Would the question also include about 


14 contracting? 


15 THE COURT: You know, I'm having trouble with hearing 


A} . NEMTSEV: 'm sorry. Would the question also 


18 include contracting with the Russian Federation? 


19 THE COURT: I don't know. 


01:17 20 MR. NEMTSEV: t implies closer ties than just being a 


21 Russian national. That's the issue. 


22 THE COURT: I don't know. I don't know. If you leave 


23 it open like that, for all they know, it's providing military 


24 equipment, you know, or that he's part of the 2016 issue. I 


25 mean, I just —- 


01:17 10 
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MR. 


NEMTSEV: And that was one of the exhibits your 


Honor excluded that implicated -—- 


THE 


THE 


THE 


now unless 


CO : All right, thank you. 


CLERK: No other date? We're good. 


COURT: We're not going to have another date right 


—- but if you need me, you've got to do it by next 


Wednesday, okay? I'm out Thursday and Friday. 


MR. 
issue is the 
obviously on 

THE 
this totally 


contested, an 


T'll meet you 


01:18 20 


21 


22 


23 


24 


25 


contested exh 


it at 8:30 in 


NEMTSEV: So, your Honor, the only foreseeable 


exhibits. There's going to be objections 


both sides, I presume, so —— 


COURT: Yes. Maryellen has it very firm. I give 


over to her. We want the exhibits that are not 


d we want the exhibits that are contested. And 


every morning at 8:30, and we'll go through the 


ibits, so if there's a big issue, we'll deal with 


the morning. 


MR. 


MR. 


NEMTSEV: Thank you, your Honor. 


FRANK: Can we just briefly address the time on 


opening statements? 


THE 


COURT: One to three days? 


(Laughter. 


MR. 


THE 


MR. 


45 minutes. 


FRANK: Even I would get sick of hearing myself. 


COURT: Well, what do you want? 


FRANK: I don't know exactly but probably around 
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THE COURT: Fine. What do you want? 


MR. NEMTSEV: Half an hour probably. 


THE COURT: Yes, fine, totally in the norm. Okay, 


thank you. 


(Adjourned, 1:18 p.m.) 
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